* [PATCH 06/17] all: replace nodes_weight with nodes_empty where appropriate
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Kernel code calls nodes_weight() to check if any bit of a given nodemask is
set. We can do it more efficiently with nodes_empty() because nodes_empty()
stops traversing the nodemask as soon as it finds first set bit, while
nodes_weight() counts all bits unconditionally.
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
arch/x86/mm/amdtopology.c | 2 +-
arch/x86/mm/numa_emulation.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/x86/mm/amdtopology.c b/arch/x86/mm/amdtopology.c
index 058b2f36b3a6..b3ca7d23e4b0 100644
--- a/arch/x86/mm/amdtopology.c
+++ b/arch/x86/mm/amdtopology.c
@@ -154,7 +154,7 @@ int __init amd_numa_init(void)
node_set(nodeid, numa_nodes_parsed);
}
- if (!nodes_weight(numa_nodes_parsed))
+ if (nodes_empty(numa_nodes_parsed))
return -ENOENT;
/*
diff --git a/arch/x86/mm/numa_emulation.c b/arch/x86/mm/numa_emulation.c
index 1a02b791d273..9a9305367fdd 100644
--- a/arch/x86/mm/numa_emulation.c
+++ b/arch/x86/mm/numa_emulation.c
@@ -123,7 +123,7 @@ static int __init split_nodes_interleave(struct numa_meminfo *ei,
* Continue to fill physical nodes with fake nodes until there is no
* memory left on any of them.
*/
- while (nodes_weight(physnode_mask)) {
+ while (!nodes_empty(physnode_mask)) {
for_each_node_mask(i, physnode_mask) {
u64 dma32_end = PFN_PHYS(MAX_DMA32_PFN);
u64 start, limit, end;
@@ -270,7 +270,7 @@ static int __init split_nodes_size_interleave_uniform(struct numa_meminfo *ei,
* Fill physical nodes with fake nodes of size until there is no memory
* left on any of them.
*/
- while (nodes_weight(physnode_mask)) {
+ while (!nodes_empty(physnode_mask)) {
for_each_node_mask(i, physnode_mask) {
u64 dma32_end = PFN_PHYS(MAX_DMA32_PFN);
u64 start, limit, end;
--
2.30.2
^ permalink raw reply related
* [PATCH 07/17] lib/bitmap: add bitmap_weight_{cmp, eq, gt, ge, lt, le} functions
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Many kernel users use bitmap_weight() to compare the result against
some number or expression:
if (bitmap_weight(...) > 1)
do_something();
It works OK, but may be significantly improved for large bitmaps: if
first few words count set bits to a number greater than given, we can
stop counting and immediately return.
The same idea would work in other direction: if we know that the number
of set bits that we counted so far is small enough, so that it would be
smaller than required number even if all bits of the rest of the bitmap
are set, we can stop counting earlier.
This patch adds new bitmap_weight_cmp() as suggested by Michał Mirosław
and a family of eq, gt, ge, lt and le wrappers to allow this optimization.
The following patches apply new functions where appropriate.
Suggested-by: "Michał Mirosław" <mirq-linux@rere.qmqm.pl> (for bitmap_weight_cmp)
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
include/linux/bitmap.h | 80 ++++++++++++++++++++++++++++++++++++++++++
lib/bitmap.c | 21 +++++++++++
2 files changed, 101 insertions(+)
diff --git a/include/linux/bitmap.h b/include/linux/bitmap.h
index 7dba0847510c..708e57b32362 100644
--- a/include/linux/bitmap.h
+++ b/include/linux/bitmap.h
@@ -51,6 +51,12 @@ struct device;
* bitmap_empty(src, nbits) Are all bits zero in *src?
* bitmap_full(src, nbits) Are all bits set in *src?
* bitmap_weight(src, nbits) Hamming Weight: number set bits
+ * bitmap_weight_cmp(src, nbits) compare Hamming Weight with a number
+ * bitmap_weight_eq(src, nbits, num) Hamming Weight == num
+ * bitmap_weight_gt(src, nbits, num) Hamming Weight > num
+ * bitmap_weight_ge(src, nbits, num) Hamming Weight >= num
+ * bitmap_weight_lt(src, nbits, num) Hamming Weight < num
+ * bitmap_weight_le(src, nbits, num) Hamming Weight <= num
* bitmap_set(dst, pos, nbits) Set specified bit area
* bitmap_clear(dst, pos, nbits) Clear specified bit area
* bitmap_find_next_zero_area(buf, len, pos, n, mask) Find bit free area
@@ -162,6 +168,7 @@ int __bitmap_intersects(const unsigned long *bitmap1,
int __bitmap_subset(const unsigned long *bitmap1,
const unsigned long *bitmap2, unsigned int nbits);
int __bitmap_weight(const unsigned long *bitmap, unsigned int nbits);
+int __bitmap_weight_cmp(const unsigned long *bitmap, unsigned int bits, int num);
void __bitmap_set(unsigned long *map, unsigned int start, int len);
void __bitmap_clear(unsigned long *map, unsigned int start, int len);
@@ -403,6 +410,79 @@ static __always_inline int bitmap_weight(const unsigned long *src, unsigned int
return __bitmap_weight(src, nbits);
}
+/**
+ * bitmap_weight_cmp - compares number of set bits in @src with @num.
+ * @src: source bitmap
+ * @nbits: length of bitmap in bits
+ * @num: number to compare with
+ *
+ * As opposite to bitmap_weight() this function doesn't necessarily
+ * traverse full bitmap and may return earlier.
+ *
+ * Returns zero if weight of @src is equal to @num;
+ * negative number if weight of @src is less than @num;
+ * positive number if weight of @src is greater than @num;
+ *
+ * NOTES
+ *
+ * Because number of set bits cannot decrease while counting, when user
+ * wants to know if the number of set bits in the bitmap is less than
+ * @num, calling
+ * bitmap_weight_cmp(..., @num) < 0
+ * is potentially less effective than
+ * bitmap_weight_cmp(..., @num - 1) <= 0
+ *
+ * Consider an example:
+ * bitmap_weight_cmp(1000 0000 0000 0000, 1) < 0
+ * ^
+ * stop here
+ *
+ * bitmap_weight_cmp(1000 0000 0000 0000, 0) <= 0
+ * ^
+ * stop here
+ */
+static __always_inline
+int bitmap_weight_cmp(const unsigned long *src, unsigned int nbits, int num)
+{
+ if (num > (int)nbits || num < 0)
+ return -num;
+
+ if (small_const_nbits(nbits))
+ return hweight_long(*src & BITMAP_LAST_WORD_MASK(nbits)) - num;
+
+ return __bitmap_weight_cmp(src, nbits, num);
+}
+
+static __always_inline
+bool bitmap_weight_eq(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num) == 0;
+}
+
+static __always_inline
+bool bitmap_weight_gt(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num) > 0;
+}
+
+static __always_inline
+bool bitmap_weight_ge(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num - 1) > 0;
+}
+
+static __always_inline
+bool bitmap_weight_lt(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num - 1) <= 0;
+}
+
+static __always_inline
+bool bitmap_weight_le(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num) <= 0;
+}
+
static __always_inline void bitmap_set(unsigned long *map, unsigned int start,
unsigned int nbits)
{
diff --git a/lib/bitmap.c b/lib/bitmap.c
index 926408883456..fb84ca70c5d9 100644
--- a/lib/bitmap.c
+++ b/lib/bitmap.c
@@ -348,6 +348,27 @@ int __bitmap_weight(const unsigned long *bitmap, unsigned int bits)
}
EXPORT_SYMBOL(__bitmap_weight);
+int __bitmap_weight_cmp(const unsigned long *bitmap, unsigned int bits, int num)
+{
+ unsigned int k, w, lim = bits / BITS_PER_LONG;
+
+ for (k = 0, w = 0; k < lim; k++) {
+ if (w + bits - k * BITS_PER_LONG < num)
+ goto out;
+
+ w += hweight_long(bitmap[k]);
+
+ if (w > num)
+ goto out;
+ }
+
+ if (bits % BITS_PER_LONG)
+ w += hweight_long(bitmap[k] & BITMAP_LAST_WORD_MASK(bits));
+out:
+ return w - num;
+}
+EXPORT_SYMBOL(__bitmap_weight_cmp);
+
void __bitmap_set(unsigned long *map, unsigned int start, int len)
{
unsigned long *p = map + BIT_WORD(start);
--
2.30.2
^ permalink raw reply related
* [PATCH 08/17] all: replace bitmap_weight with bitmap_weight_{eq, gt, ge, lt, le} where appropriate
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Kernel code calls bitmap_weight() to compare the weight of bitmap with
a given number. We can do it more efficiently with bitmap_weight_{eq, ...}
because conditional bitmap_weight may stop traversing the bitmap earlier,
as soon as condition is met.
This patch replaces bitmap_weight with conditional versions where possible,
except for small bitmaps which size is not configurable and known at
constant time. In that case conditional version of bitmap_weight would not
benefit due to small_const_nbits() optimization; but readability may
suffer.
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 +-
drivers/iio/dummy/iio_simple_dummy_buffer.c | 4 ++--
drivers/iio/industrialio-trigger.c | 2 +-
drivers/memstick/core/ms_block.c | 4 ++--
drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 2 +-
.../net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c | 2 +-
.../net/ethernet/marvell/octeontx2/nic/otx2_flows.c | 4 ++--
drivers/net/ethernet/mellanox/mlx4/cmd.c | 10 +++-------
drivers/net/ethernet/mellanox/mlx4/eq.c | 4 ++--
drivers/net/ethernet/mellanox/mlx4/fw.c | 4 ++--
drivers/net/ethernet/mellanox/mlx4/main.c | 2 +-
drivers/perf/thunderx2_pmu.c | 4 ++--
drivers/staging/media/tegra-video/vi.c | 2 +-
13 files changed, 21 insertions(+), 25 deletions(-)
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index e23ff03290b8..9d42e592c1cf 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -2752,7 +2752,7 @@ static int __init_one_rdt_domain(struct rdt_domain *d, struct resctrl_schema *s,
* bitmap_weight() does not access out-of-bound memory.
*/
tmp_cbm = cfg->new_ctrl;
- if (bitmap_weight(&tmp_cbm, r->cache.cbm_len) < r->cache.min_cbm_bits) {
+ if (bitmap_weight_lt(&tmp_cbm, r->cache.cbm_len, r->cache.min_cbm_bits)) {
rdt_last_cmd_printf("No space on %s:%d\n", s->name, d->id);
return -ENOSPC;
}
diff --git a/drivers/iio/dummy/iio_simple_dummy_buffer.c b/drivers/iio/dummy/iio_simple_dummy_buffer.c
index 59aa60d4ca37..cd2470ddf82b 100644
--- a/drivers/iio/dummy/iio_simple_dummy_buffer.c
+++ b/drivers/iio/dummy/iio_simple_dummy_buffer.c
@@ -72,8 +72,8 @@ static irqreturn_t iio_simple_dummy_trigger_h(int irq, void *p)
int i, j;
for (i = 0, j = 0;
- i < bitmap_weight(indio_dev->active_scan_mask,
- indio_dev->masklength);
+ bitmap_weight_gt(indio_dev->active_scan_mask,
+ indio_dev->masklength, i);
i++, j++) {
j = find_next_bit(indio_dev->active_scan_mask,
indio_dev->masklength, j);
diff --git a/drivers/iio/industrialio-trigger.c b/drivers/iio/industrialio-trigger.c
index f504ed351b3e..98c54022fecf 100644
--- a/drivers/iio/industrialio-trigger.c
+++ b/drivers/iio/industrialio-trigger.c
@@ -331,7 +331,7 @@ int iio_trigger_detach_poll_func(struct iio_trigger *trig,
{
struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(pf->indio_dev);
bool no_other_users =
- bitmap_weight(trig->pool, CONFIG_IIO_CONSUMERS_PER_TRIGGER) == 1;
+ bitmap_weight_eq(trig->pool, CONFIG_IIO_CONSUMERS_PER_TRIGGER, 1);
int ret = 0;
if (trig->ops && trig->ops->set_trigger_state && no_other_users) {
diff --git a/drivers/memstick/core/ms_block.c b/drivers/memstick/core/ms_block.c
index 0cda6c6baefc..5cdd987e78f7 100644
--- a/drivers/memstick/core/ms_block.c
+++ b/drivers/memstick/core/ms_block.c
@@ -155,8 +155,8 @@ static int msb_validate_used_block_bitmap(struct msb_data *msb)
for (i = 0; i < msb->zone_count; i++)
total_free_blocks += msb->free_block_count[i];
- if (msb->block_count - bitmap_weight(msb->used_blocks_bitmap,
- msb->block_count) == total_free_blocks)
+ if (bitmap_weight_eq(msb->used_blocks_bitmap, msb->block_count,
+ msb->block_count - total_free_blocks))
return 0;
pr_err("BUG: free block counts don't match the bitmap");
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c
index 214a38de3f41..35297d8a488b 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c
@@ -246,7 +246,7 @@ int ixgbe_disable_sriov(struct ixgbe_adapter *adapter)
#endif
/* Disable VMDq flag so device will be set in VM mode */
- if (bitmap_weight(adapter->fwd_bitmask, adapter->num_rx_pools) == 1) {
+ if (bitmap_weight_eq(adapter->fwd_bitmask, adapter->num_rx_pools, 1)) {
adapter->flags &= ~IXGBE_FLAG_VMDQ_ENABLED;
adapter->flags &= ~IXGBE_FLAG_SRIOV_ENABLED;
rss = min_t(int, ixgbe_max_rss_indices(adapter),
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
index d85db90632d6..a55fd1d0c653 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
@@ -287,7 +287,7 @@ static int otx2_set_channels(struct net_device *dev,
if (!channel->rx_count || !channel->tx_count)
return -EINVAL;
- if (bitmap_weight(&pfvf->rq_bmap, pfvf->hw.rx_queues) > 1) {
+ if (bitmap_weight_gt(&pfvf->rq_bmap, pfvf->hw.rx_queues, 1)) {
netdev_err(dev,
"Receive queues are in use by TC police action\n");
return -EINVAL;
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c
index 80b2d64b4136..55c899a6fcdd 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c
@@ -1170,8 +1170,8 @@ int otx2_remove_flow(struct otx2_nic *pfvf, u32 location)
* interface mac address and configure CGX/RPM block in
* promiscuous mode
*/
- if (bitmap_weight(&flow_cfg->dmacflt_bmap,
- flow_cfg->dmacflt_max_flows) == 1)
+ if (bitmap_weight_eq(&flow_cfg->dmacflt_bmap,
+ flow_cfg->dmacflt_max_flows, 1))
otx2_update_rem_pfmac(pfvf, DMAC_ADDR_DEL);
} else {
err = otx2_remove_flow_msg(pfvf, flow->entry, false);
diff --git a/drivers/net/ethernet/mellanox/mlx4/cmd.c b/drivers/net/ethernet/mellanox/mlx4/cmd.c
index c56d2194cbfc..5bca0c68f00a 100644
--- a/drivers/net/ethernet/mellanox/mlx4/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx4/cmd.c
@@ -2792,9 +2792,8 @@ int mlx4_slave_convert_port(struct mlx4_dev *dev, int slave, int port)
{
unsigned n;
struct mlx4_active_ports actv_ports = mlx4_get_active_ports(dev, slave);
- unsigned m = bitmap_weight(actv_ports.ports, dev->caps.num_ports);
- if (port <= 0 || port > m)
+ if (port <= 0 || bitmap_weight_lt(actv_ports.ports, dev->caps.num_ports, port))
return -EINVAL;
n = find_first_bit(actv_ports.ports, dev->caps.num_ports);
@@ -3404,10 +3403,6 @@ int mlx4_vf_set_enable_smi_admin(struct mlx4_dev *dev, int slave, int port,
struct mlx4_priv *priv = mlx4_priv(dev);
struct mlx4_active_ports actv_ports = mlx4_get_active_ports(
&priv->dev, slave);
- int min_port = find_first_bit(actv_ports.ports,
- priv->dev.caps.num_ports) + 1;
- int max_port = min_port - 1 +
- bitmap_weight(actv_ports.ports, priv->dev.caps.num_ports);
if (slave == mlx4_master_func_num(dev))
return 0;
@@ -3417,7 +3412,8 @@ int mlx4_vf_set_enable_smi_admin(struct mlx4_dev *dev, int slave, int port,
enabled < 0 || enabled > 1)
return -EINVAL;
- if (min_port == max_port && dev->caps.num_ports > 1) {
+ if (dev->caps.num_ports > 1 &&
+ bitmap_weight_eq(actv_ports.ports, priv->dev.caps.num_ports, 1)) {
mlx4_info(dev, "SMI access disallowed for single ported VFs\n");
return -EPROTONOSUPPORT;
}
diff --git a/drivers/net/ethernet/mellanox/mlx4/eq.c b/drivers/net/ethernet/mellanox/mlx4/eq.c
index 414e390e6b48..0c09432ff389 100644
--- a/drivers/net/ethernet/mellanox/mlx4/eq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/eq.c
@@ -1435,8 +1435,8 @@ int mlx4_is_eq_shared(struct mlx4_dev *dev, int vector)
if (vector <= 0 || (vector >= dev->caps.num_comp_vectors + 1))
return -EINVAL;
- return !!(bitmap_weight(priv->eq_table.eq[vector].actv_ports.ports,
- dev->caps.num_ports) > 1);
+ return bitmap_weight_gt(priv->eq_table.eq[vector].actv_ports.ports,
+ dev->caps.num_ports, 1);
}
EXPORT_SYMBOL(mlx4_is_eq_shared);
diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c b/drivers/net/ethernet/mellanox/mlx4/fw.c
index 42c96c9d7fb1..855aae326ccb 100644
--- a/drivers/net/ethernet/mellanox/mlx4/fw.c
+++ b/drivers/net/ethernet/mellanox/mlx4/fw.c
@@ -1300,8 +1300,8 @@ int mlx4_QUERY_DEV_CAP_wrapper(struct mlx4_dev *dev, int slave,
actv_ports = mlx4_get_active_ports(dev, slave);
first_port = find_first_bit(actv_ports.ports, dev->caps.num_ports);
for (slave_port = 0, real_port = first_port;
- real_port < first_port +
- bitmap_weight(actv_ports.ports, dev->caps.num_ports);
+ bitmap_weight_gt(actv_ports.ports, dev->caps.num_ports,
+ real_port - first_port);
++real_port, ++slave_port) {
if (flags & (MLX4_DEV_CAP_FLAG_WOL_PORT1 << real_port))
flags |= MLX4_DEV_CAP_FLAG_WOL_PORT1 << slave_port;
diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index b187c210d4d6..cfbaa7ac712f 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -1383,7 +1383,7 @@ static int mlx4_mf_bond(struct mlx4_dev *dev)
dev->persist->num_vfs + 1);
/* only single port vfs are allowed */
- if (bitmap_weight(slaves_port_1_2, dev->persist->num_vfs + 1) > 1) {
+ if (bitmap_weight_gt(slaves_port_1_2, dev->persist->num_vfs + 1, 1)) {
mlx4_warn(dev, "HA mode unsupported for dual ported VFs\n");
return -EINVAL;
}
diff --git a/drivers/perf/thunderx2_pmu.c b/drivers/perf/thunderx2_pmu.c
index 05378c0fd8f3..ebfa66b212c7 100644
--- a/drivers/perf/thunderx2_pmu.c
+++ b/drivers/perf/thunderx2_pmu.c
@@ -623,8 +623,8 @@ static void tx2_uncore_event_start(struct perf_event *event, int flags)
return;
/* Start timer for first event */
- if (bitmap_weight(tx2_pmu->active_counters,
- tx2_pmu->max_counters) == 1) {
+ if (bitmap_weight_eq(tx2_pmu->active_counters,
+ tx2_pmu->max_counters, 1)) {
hrtimer_start(&tx2_pmu->hrtimer,
ns_to_ktime(tx2_pmu->hrtimer_interval),
HRTIMER_MODE_REL_PINNED);
diff --git a/drivers/staging/media/tegra-video/vi.c b/drivers/staging/media/tegra-video/vi.c
index 69d9787d5338..98d878a5ca6b 100644
--- a/drivers/staging/media/tegra-video/vi.c
+++ b/drivers/staging/media/tegra-video/vi.c
@@ -436,7 +436,7 @@ static int tegra_channel_enum_format(struct file *file, void *fh,
if (!IS_ENABLED(CONFIG_VIDEO_TEGRA_TPG))
fmts_bitmap = chan->fmts_bitmap;
- if (f->index >= bitmap_weight(fmts_bitmap, MAX_FORMAT_NUM))
+ if (bitmap_weight_le(fmts_bitmap, MAX_FORMAT_NUM, f->index))
return -EINVAL;
for (i = 0; i < f->index + 1; i++, index++)
--
2.30.2
^ permalink raw reply related
* [PATCH 09/17] lib/cpumask: add cpumask_weight_{eq,gt,ge,lt,le}
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Kernel code calls cpumask_weight() to compare the weight of cpumask with
a given number. We can do it more efficiently with cpumask_weight_{eq, ...}
because conditional cpumask_weight may stop traversing the cpumask earlier,
as soon as condition is met.
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
arch/ia64/mm/tlb.c | 2 +-
arch/mips/cavium-octeon/octeon-irq.c | 4 +-
arch/mips/kernel/crash.c | 2 +-
arch/powerpc/kernel/smp.c | 2 +-
arch/powerpc/kernel/watchdog.c | 2 +-
arch/powerpc/xmon/xmon.c | 4 +-
arch/s390/kernel/perf_cpum_cf.c | 2 +-
arch/x86/kernel/smpboot.c | 4 +-
drivers/firmware/psci/psci_checker.c | 2 +-
drivers/hv/channel_mgmt.c | 4 +-
drivers/infiniband/hw/hfi1/affinity.c | 9 ++---
drivers/infiniband/hw/qib/qib_file_ops.c | 2 +-
drivers/infiniband/hw/qib/qib_iba7322.c | 2 +-
drivers/scsi/lpfc/lpfc_init.c | 2 +-
drivers/soc/fsl/qbman/qman_test_stash.c | 2 +-
include/linux/cpumask.h | 50 ++++++++++++++++++++++++
kernel/sched/core.c | 8 ++--
kernel/sched/topology.c | 2 +-
kernel/time/clockevents.c | 2 +-
19 files changed, 78 insertions(+), 29 deletions(-)
diff --git a/arch/ia64/mm/tlb.c b/arch/ia64/mm/tlb.c
index 135b5135cace..a5bce13ab047 100644
--- a/arch/ia64/mm/tlb.c
+++ b/arch/ia64/mm/tlb.c
@@ -332,7 +332,7 @@ __flush_tlb_range (struct vm_area_struct *vma, unsigned long start,
preempt_disable();
#ifdef CONFIG_SMP
- if (mm != current->active_mm || cpumask_weight(mm_cpumask(mm)) != 1) {
+ if (mm != current->active_mm || !cpumask_weight_eq(mm_cpumask(mm), 1)) {
ia64_global_tlb_purge(mm, start, end, nbits);
preempt_enable();
return;
diff --git a/arch/mips/cavium-octeon/octeon-irq.c b/arch/mips/cavium-octeon/octeon-irq.c
index 844f882096e6..914871f15fb7 100644
--- a/arch/mips/cavium-octeon/octeon-irq.c
+++ b/arch/mips/cavium-octeon/octeon-irq.c
@@ -763,7 +763,7 @@ static void octeon_irq_cpu_offline_ciu(struct irq_data *data)
if (!cpumask_test_cpu(cpu, mask))
return;
- if (cpumask_weight(mask) > 1) {
+ if (cpumask_weight_gt(mask, 1)) {
/*
* It has multi CPU affinity, just remove this CPU
* from the affinity set.
@@ -795,7 +795,7 @@ static int octeon_irq_ciu_set_affinity(struct irq_data *data,
* This removes the need to do locking in the .ack/.eoi
* functions.
*/
- if (cpumask_weight(dest) != 1)
+ if (!cpumask_weight_eq(dest, 1))
return -EINVAL;
if (!enable_one)
diff --git a/arch/mips/kernel/crash.c b/arch/mips/kernel/crash.c
index 81845ba04835..5b690d52491f 100644
--- a/arch/mips/kernel/crash.c
+++ b/arch/mips/kernel/crash.c
@@ -72,7 +72,7 @@ static void crash_kexec_prepare_cpus(void)
*/
pr_emerg("Sending IPI to other cpus...\n");
msecs = 10000;
- while ((cpumask_weight(&cpus_in_crash) < ncpus) && (--msecs > 0)) {
+ while (cpumask_weight_lt(&cpus_in_crash, ncpus) && (--msecs > 0)) {
cpu_relax();
mdelay(1);
}
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index c338f9d8ab37..00da2064ddf3 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1655,7 +1655,7 @@ void start_secondary(void *unused)
if (has_big_cores)
sibling_mask = cpu_smallcore_mask;
- if (cpumask_weight(mask) > cpumask_weight(sibling_mask(cpu)))
+ if (cpumask_weight_gt(mask, cpumask_weight(sibling_mask(cpu))))
shared_caches = true;
}
diff --git a/arch/powerpc/kernel/watchdog.c b/arch/powerpc/kernel/watchdog.c
index bfc27496fe7e..62937a077de7 100644
--- a/arch/powerpc/kernel/watchdog.c
+++ b/arch/powerpc/kernel/watchdog.c
@@ -483,7 +483,7 @@ static void start_watchdog(void *arg)
wd_smp_lock(&flags);
cpumask_set_cpu(cpu, &wd_cpus_enabled);
- if (cpumask_weight(&wd_cpus_enabled) == 1) {
+ if (cpumask_weight_eq(&wd_cpus_enabled, 1)) {
cpumask_set_cpu(cpu, &wd_smp_cpus_pending);
wd_smp_last_reset_tb = get_tb();
}
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index f9ae0b398260..b9e9d0b20a7b 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -469,7 +469,7 @@ static bool wait_for_other_cpus(int ncpus)
/* We wait for 2s, which is a metric "little while" */
for (timeout = 20000; timeout != 0; --timeout) {
- if (cpumask_weight(&cpus_in_xmon) >= ncpus)
+ if (cpumask_weight_ge(&cpus_in_xmon, ncpus))
return true;
udelay(100);
barrier();
@@ -1338,7 +1338,7 @@ static int cpu_cmd(void)
case 'S':
case 't':
cpumask_copy(&xmon_batch_cpus, &cpus_in_xmon);
- if (cpumask_weight(&xmon_batch_cpus) <= 1) {
+ if (cpumask_weight_le(&xmon_batch_cpus, 1)) {
printf("There are no other cpus in xmon\n");
break;
}
diff --git a/arch/s390/kernel/perf_cpum_cf.c b/arch/s390/kernel/perf_cpum_cf.c
index ee8707abdb6a..4d217f7f5ccf 100644
--- a/arch/s390/kernel/perf_cpum_cf.c
+++ b/arch/s390/kernel/perf_cpum_cf.c
@@ -975,7 +975,7 @@ static int cfset_all_start(struct cfset_request *req)
return -ENOMEM;
cpumask_and(mask, &req->mask, cpu_online_mask);
on_each_cpu_mask(mask, cfset_ioctl_on, &p, 1);
- if (atomic_read(&p.cpus_ack) != cpumask_weight(mask)) {
+ if (!cpumask_weight_eq(mask, atomic_read(&p.cpus_ack))) {
on_each_cpu_mask(mask, cfset_ioctl_off, &p, 1);
rc = -EIO;
debug_sprintf_event(cf_dbg, 4, "%s CPUs missing", __func__);
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 617012f4619f..e851e9945eb5 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1608,7 +1608,7 @@ static void remove_siblinginfo(int cpu)
/*/
* last thread sibling in this cpu core going down
*/
- if (cpumask_weight(topology_sibling_cpumask(cpu)) == 1)
+ if (cpumask_weight_eq(topology_sibling_cpumask(cpu), 1))
cpu_data(sibling).booted_cores--;
}
@@ -1617,7 +1617,7 @@ static void remove_siblinginfo(int cpu)
for_each_cpu(sibling, topology_sibling_cpumask(cpu)) {
cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
- if (cpumask_weight(topology_sibling_cpumask(sibling)) == 1)
+ if (cpumask_weight_eq(topology_sibling_cpumask(sibling), 1))
cpu_data(sibling).smt_active = false;
}
diff --git a/drivers/firmware/psci/psci_checker.c b/drivers/firmware/psci/psci_checker.c
index 116eb465cdb4..90c9473832a9 100644
--- a/drivers/firmware/psci/psci_checker.c
+++ b/drivers/firmware/psci/psci_checker.c
@@ -90,7 +90,7 @@ static unsigned int down_and_up_cpus(const struct cpumask *cpus,
* cpu_down() checks the number of online CPUs before the TOS
* resident CPU.
*/
- if (cpumask_weight(offlined_cpus) + 1 == nb_available_cpus) {
+ if (cpumask_weight_eq(offlined_cpus, nb_available_cpus - 1)) {
if (ret != -EBUSY) {
pr_err("Unexpected return code %d while trying "
"to power down last online CPU %d\n",
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 2829575fd9b7..da297220230d 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -762,8 +762,8 @@ static void init_vp_index(struct vmbus_channel *channel)
}
alloced_mask = &hv_context.hv_numa_map[numa_node];
- if (cpumask_weight(alloced_mask) ==
- cpumask_weight(cpumask_of_node(numa_node))) {
+ if (cpumask_weight_eq(alloced_mask,
+ cpumask_weight(cpumask_of_node(numa_node)))) {
/*
* We have cycled through all the CPUs in the node;
* reset the alloced map.
diff --git a/drivers/infiniband/hw/hfi1/affinity.c b/drivers/infiniband/hw/hfi1/affinity.c
index 38eee675369a..7c5ca5c5306a 100644
--- a/drivers/infiniband/hw/hfi1/affinity.c
+++ b/drivers/infiniband/hw/hfi1/affinity.c
@@ -507,7 +507,7 @@ static int _dev_comp_vect_cpu_mask_init(struct hfi1_devdata *dd,
* available CPUs divide it by the number of devices in the
* local NUMA node.
*/
- if (cpumask_weight(&entry->comp_vect_mask) == 1) {
+ if (cpumask_weight_eq(&entry->comp_vect_mask, 1)) {
possible_cpus_comp_vect = 1;
dd_dev_warn(dd,
"Number of kernel receive queues is too large for completion vector affinity to be effective\n");
@@ -593,7 +593,7 @@ int hfi1_dev_affinity_init(struct hfi1_devdata *dd)
{
struct hfi1_affinity_node *entry;
const struct cpumask *local_mask;
- int curr_cpu, possible, i, ret;
+ int curr_cpu, i, ret;
bool new_entry = false;
local_mask = cpumask_of_node(dd->node);
@@ -626,10 +626,9 @@ int hfi1_dev_affinity_init(struct hfi1_devdata *dd)
local_mask);
/* fill in the receive list */
- possible = cpumask_weight(&entry->def_intr.mask);
curr_cpu = cpumask_first(&entry->def_intr.mask);
- if (possible == 1) {
+ if (cpumask_weight_eq(&entry->def_intr.mask, 1)) {
/* only one CPU, everyone will use it */
cpumask_set_cpu(curr_cpu, &entry->rcv_intr.mask);
cpumask_set_cpu(curr_cpu, &entry->general_intr_mask);
@@ -1017,7 +1016,7 @@ int hfi1_get_proc_affinity(int node)
cpu = cpumask_first(proc_mask);
cpumask_set_cpu(cpu, &set->used);
goto done;
- } else if (current->nr_cpus_allowed < cpumask_weight(&set->mask)) {
+ } else if (cpumask_weight_gt(&set->mask, current->nr_cpus_allowed)) {
hfi1_cdbg(PROC, "PID %u %s affinity set to CPU set(s) %*pbl",
current->pid, current->comm,
cpumask_pr_args(proc_mask));
diff --git a/drivers/infiniband/hw/qib/qib_file_ops.c b/drivers/infiniband/hw/qib/qib_file_ops.c
index aa290928cf96..add89bc21b0a 100644
--- a/drivers/infiniband/hw/qib/qib_file_ops.c
+++ b/drivers/infiniband/hw/qib/qib_file_ops.c
@@ -1151,7 +1151,7 @@ static void assign_ctxt_affinity(struct file *fp, struct qib_devdata *dd)
* reserve a processor for it on the local NUMA node.
*/
if ((weight >= qib_cpulist_count) &&
- (cpumask_weight(local_mask) <= qib_cpulist_count)) {
+ (cpumask_weight_le(local_mask, qib_cpulist_count))) {
for_each_cpu(local_cpu, local_mask)
if (!test_and_set_bit(local_cpu, qib_cpulist)) {
fd->rec_cpu_num = local_cpu;
diff --git a/drivers/infiniband/hw/qib/qib_iba7322.c b/drivers/infiniband/hw/qib/qib_iba7322.c
index ab98b6a3ae1e..636a080b2952 100644
--- a/drivers/infiniband/hw/qib/qib_iba7322.c
+++ b/drivers/infiniband/hw/qib/qib_iba7322.c
@@ -3405,7 +3405,7 @@ static void qib_setup_7322_interrupt(struct qib_devdata *dd, int clearpend)
local_mask = cpumask_of_pcibus(dd->pcidev->bus);
firstcpu = cpumask_first(local_mask);
if (firstcpu >= nr_cpu_ids ||
- cpumask_weight(local_mask) == num_online_cpus()) {
+ cpumask_weight_eq(local_mask, num_online_cpus())) {
local_mask = topology_core_cpumask(0);
firstcpu = cpumask_first(local_mask);
}
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index a56f01f659f8..325e9004dacd 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -12643,7 +12643,7 @@ lpfc_cpuhp_get_eq(struct lpfc_hba *phba, unsigned int cpu,
* gone offline yet, we need >1.
*/
cpumask_and(tmp, maskp, cpu_online_mask);
- if (cpumask_weight(tmp) > 1)
+ if (cpumask_weight_gt(tmp, 1))
continue;
/* Now that we have an irq to shutdown, get the eq
diff --git a/drivers/soc/fsl/qbman/qman_test_stash.c b/drivers/soc/fsl/qbman/qman_test_stash.c
index b7e8e5ec884c..28b08568a349 100644
--- a/drivers/soc/fsl/qbman/qman_test_stash.c
+++ b/drivers/soc/fsl/qbman/qman_test_stash.c
@@ -561,7 +561,7 @@ int qman_test_stash(void)
{
int err;
- if (cpumask_weight(cpu_online_mask) < 2) {
+ if (cpumask_weight_lt(cpu_online_mask, 2)) {
pr_info("%s(): skip - only 1 CPU\n", __func__);
return 0;
}
diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h
index 64dae70d31f5..1906e3225737 100644
--- a/include/linux/cpumask.h
+++ b/include/linux/cpumask.h
@@ -575,6 +575,56 @@ static inline unsigned int cpumask_weight(const struct cpumask *srcp)
return bitmap_weight(cpumask_bits(srcp), nr_cpumask_bits);
}
+/**
+ * cpumask_weight_eq - Check if # of bits in *srcp is equal to a given number
+ * @srcp: the cpumask to count bits (< nr_cpu_ids) in.
+ * @num: the number to check.
+ */
+static inline bool cpumask_weight_eq(const struct cpumask *srcp, unsigned int num)
+{
+ return bitmap_weight_eq(cpumask_bits(srcp), nr_cpumask_bits, num);
+}
+
+/**
+ * cpumask_weight_gt - Check if # of bits in *srcp is greater than a given number
+ * @srcp: the cpumask to count bits (< nr_cpu_ids) in.
+ * @num: the number to check.
+ */
+static inline bool cpumask_weight_gt(const struct cpumask *srcp, int num)
+{
+ return bitmap_weight_gt(cpumask_bits(srcp), nr_cpumask_bits, num);
+}
+
+/**
+ * cpumask_weight_ge - Check if # of bits in *srcp is greater than or equal to a given number
+ * @srcp: the cpumask to count bits (< nr_cpu_ids) in.
+ * @num: the number to check.
+ */
+static inline bool cpumask_weight_ge(const struct cpumask *srcp, int num)
+{
+ return bitmap_weight_ge(cpumask_bits(srcp), nr_cpumask_bits, num);
+}
+
+/**
+ * cpumask_weight_lt - Check if # of bits in *srcp is less than a given number
+ * @srcp: the cpumask to count bits (< nr_cpu_ids) in.
+ * @num: the number to check.
+ */
+static inline bool cpumask_weight_lt(const struct cpumask *srcp, int num)
+{
+ return bitmap_weight_lt(cpumask_bits(srcp), nr_cpumask_bits, num);
+}
+
+/**
+ * cpumask_weight_le - Check if # of bits in *srcp is less than or equal to a given number
+ * @srcp: the cpumask to count bits (< nr_cpu_ids) in.
+ * @num: the number to check.
+ */
+static inline bool cpumask_weight_le(const struct cpumask *srcp, int num)
+{
+ return bitmap_weight_le(cpumask_bits(srcp), nr_cpumask_bits, num);
+}
+
/**
* cpumask_shift_right - *dstp = *srcp >> n
* @dstp: the cpumask result
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 9b3ec14227e1..60f7d04a05f8 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6006,7 +6006,7 @@ static void sched_core_cpu_starting(unsigned int cpu)
WARN_ON_ONCE(rq->core != rq);
/* if we're the first, we'll be our own leader */
- if (cpumask_weight(smt_mask) == 1)
+ if (cpumask_weight_eq(smt_mask, 1))
goto unlock;
/* find the leader */
@@ -6047,7 +6047,7 @@ static void sched_core_cpu_deactivate(unsigned int cpu)
sched_core_lock(cpu, &flags);
/* if we're the last man standing, nothing to do */
- if (cpumask_weight(smt_mask) == 1) {
+ if (cpumask_weight_eq(smt_mask, 1)) {
WARN_ON_ONCE(rq->core != rq);
goto unlock;
}
@@ -9053,7 +9053,7 @@ int sched_cpu_activate(unsigned int cpu)
/*
* When going up, increment the number of cores with SMT present.
*/
- if (cpumask_weight(cpu_smt_mask(cpu)) == 2)
+ if (cpumask_weight_eq(cpu_smt_mask(cpu), 2))
static_branch_inc_cpuslocked(&sched_smt_present);
#endif
set_cpu_active(cpu, true);
@@ -9128,7 +9128,7 @@ int sched_cpu_deactivate(unsigned int cpu)
/*
* When going down, decrement the number of cores with SMT present.
*/
- if (cpumask_weight(cpu_smt_mask(cpu)) == 2)
+ if (cpumask_weight_eq(cpu_smt_mask(cpu), 2))
static_branch_dec_cpuslocked(&sched_smt_present);
sched_core_cpu_deactivate(cpu);
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 8478e2a8cd65..79395571599f 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -169,7 +169,7 @@ static const unsigned int SD_DEGENERATE_GROUPS_MASK =
static int sd_degenerate(struct sched_domain *sd)
{
- if (cpumask_weight(sched_domain_span(sd)) == 1)
+ if (cpumask_weight_eq(sched_domain_span(sd), 1))
return 1;
/* Following flags need at least 2 groups */
diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index 003ccf338d20..32d6629a55b2 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -648,7 +648,7 @@ void tick_cleanup_dead_cpu(int cpu)
*/
list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) {
if (cpumask_test_cpu(cpu, dev->cpumask) &&
- cpumask_weight(dev->cpumask) == 1 &&
+ cpumask_weight_eq(dev->cpumask, 1) &&
!tick_is_broadcast_device(dev)) {
BUG_ON(!clockevent_state_detached(dev));
list_del(&dev->list);
--
2.30.2
^ permalink raw reply related
* [PATCH 10/17] lib/nodemask: add nodemask_weight_{eq,gt,ge,lt,le}
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Kernel code calls nodes_weight() to compare the weight of nodemask with
a given number. We can do it more efficiently with nodes_weight_{eq, ...}
because conditional nodes_weight may stop traversing the nodemask earlier,
as soon as condition is met.
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
drivers/acpi/numa/srat.c | 2 +-
include/linux/nodemask.h | 35 +++++++++++++++++++++++++++++++++++
mm/mempolicy.c | 2 +-
3 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c
index 66a0142dc78c..484b9307f8cc 100644
--- a/drivers/acpi/numa/srat.c
+++ b/drivers/acpi/numa/srat.c
@@ -67,7 +67,7 @@ int acpi_map_pxm_to_node(int pxm)
node = pxm_to_node_map[pxm];
if (node == NUMA_NO_NODE) {
- if (nodes_weight(nodes_found_map) >= MAX_NUMNODES)
+ if (nodes_weight_ge(nodes_found_map, MAX_NUMNODES))
return NUMA_NO_NODE;
node = first_unset_node(nodes_found_map);
__acpi_map_pxm_to_node(pxm, node);
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 567c3ddba2c4..197598e075e9 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -38,6 +38,11 @@
* int nodes_empty(mask) Is mask empty (no bits sets)?
* int nodes_full(mask) Is mask full (all bits sets)?
* int nodes_weight(mask) Hamming weight - number of set bits
+ * bool nodes_weight_eq(src, nbits, num) Hamming Weight is equal to num
+ * bool nodes_weight_gt(src, nbits, num) Hamming Weight is greater than num
+ * bool nodes_weight_ge(src, nbits, num) Hamming Weight is greater than or equal to num
+ * bool nodes_weight_lt(src, nbits, num) Hamming Weight is less than num
+ * bool nodes_weight_le(src, nbits, num) Hamming Weight is less than or equal to num
*
* void nodes_shift_right(dst, src, n) Shift right
* void nodes_shift_left(dst, src, n) Shift left
@@ -240,6 +245,36 @@ static inline int __nodes_weight(const nodemask_t *srcp, unsigned int nbits)
return bitmap_weight(srcp->bits, nbits);
}
+#define nodes_weight_eq(nodemask, num) __nodes_weight_eq(&(nodemask), MAX_NUMNODES, (num))
+static inline int __nodes_weight_eq(const nodemask_t *srcp, unsigned int nbits, int num)
+{
+ return bitmap_weight_eq(srcp->bits, nbits, num);
+}
+
+#define nodes_weight_gt(nodemask, num) __nodes_weight_gt(&(nodemask), MAX_NUMNODES, (num))
+static inline int __nodes_weight_gt(const nodemask_t *srcp, unsigned int nbits, int num)
+{
+ return bitmap_weight_gt(srcp->bits, nbits, num);
+}
+
+#define nodes_weight_ge(nodemask, num) __nodes_weight_ge(&(nodemask), MAX_NUMNODES, (num))
+static inline int __nodes_weight_ge(const nodemask_t *srcp, unsigned int nbits, int num)
+{
+ return bitmap_weight_ge(srcp->bits, nbits, num);
+}
+
+#define nodes_weight_lt(nodemask, num) __nodes_weight_lt(&(nodemask), MAX_NUMNODES, (num))
+static inline int __nodes_weight_lt(const nodemask_t *srcp, unsigned int nbits, int num)
+{
+ return bitmap_weight_lt(srcp->bits, nbits, num);
+}
+
+#define nodes_weight_le(nodemask, num) __nodes_weight_le(&(nodemask), MAX_NUMNODES, (num))
+static inline int __nodes_weight_le(const nodemask_t *srcp, unsigned int nbits, int num)
+{
+ return bitmap_weight_le(srcp->bits, nbits, num);
+}
+
#define nodes_shift_right(dst, src, n) \
__nodes_shift_right(&(dst), &(src), (n), MAX_NUMNODES)
static inline void __nodes_shift_right(nodemask_t *dstp,
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index a86590b2507d..27817cf2f2a0 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1157,7 +1157,7 @@ int do_migrate_pages(struct mm_struct *mm, const nodemask_t *from,
* [0-7] - > [3,4,5] moves only 0,1,2,6,7.
*/
- if ((nodes_weight(*from) != nodes_weight(*to)) &&
+ if (!nodes_weight_eq(*from, nodes_weight(*to)) &&
(node_isset(s, *to)))
continue;
--
2.30.2
^ permalink raw reply related
* [PATCH 11/17] lib/nodemask: add num_node_state_eq()
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Kernel code calls num_node_state() to compare number of nodes with a given
number. The underlying code calls bitmap_weight(), and we can do it more
efficiently with num_node_state_eq because conditional nodes_weight may
stop traversing the nodemask earlier, as soon as condition is met.
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
include/linux/nodemask.h | 5 +++++
mm/page_alloc.c | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 197598e075e9..c5014dbf3cce 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -466,6 +466,11 @@ static inline int num_node_state(enum node_states state)
return nodes_weight(node_states[state]);
}
+static inline int num_node_state_eq(enum node_states state, int num)
+{
+ return nodes_weight_eq(node_states[state], num);
+}
+
#define for_each_node_state(__node, __state) \
for_each_node_mask((__node), node_states[__state])
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index edfd6c81af82..71f5652828b8 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -8323,7 +8323,7 @@ void __init page_alloc_init(void)
int ret;
#ifdef CONFIG_NUMA
- if (num_node_state(N_MEMORY) == 1)
+ if (num_node_state_eq(N_MEMORY, 1))
hashdist = 0;
#endif
--
2.30.2
^ permalink raw reply related
* [PATCH 12/17] kernel/cpu.c: fix init_cpu_online
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
cpu_online_mask has an associate counter of online cpus, which should be
initialized in init_cpu_online()
Fixes: 0c09ab96fc82010 (cpu/hotplug: Cache number of online CPUs)
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
kernel/cpu.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 407a2568f35e..cd7605204d4d 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -2616,6 +2616,7 @@ void init_cpu_possible(const struct cpumask *src)
void init_cpu_online(const struct cpumask *src)
{
cpumask_copy(&__cpu_online_mask, src);
+ atomic_set(&__num_online_cpus, cpumask_weight(cpu_online_mask));
}
void set_cpu_online(unsigned int cpu, bool online)
--
2.30.2
^ permalink raw reply related
* [PATCH 13/17] kernel/cpu: add num_possible_cpus counter
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Similarly to the online cpus, the cpu_possible_mask is actively used
in the kernel. This patch adds a counter for possible cpus, so that
users that call num_possible_cpus() would know the result immediately,
instead of calling the bitmap_weight for the mask underlying.
Suggested-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
include/linux/cpumask.h | 30 ++++++++++++++++--------------
kernel/cpu.c | 22 ++++++++++++++++++++++
2 files changed, 38 insertions(+), 14 deletions(-)
diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h
index 1906e3225737..0be2504d8e4c 100644
--- a/include/linux/cpumask.h
+++ b/include/linux/cpumask.h
@@ -99,6 +99,7 @@ extern struct cpumask __cpu_dying_mask;
#define cpu_dying_mask ((const struct cpumask *)&__cpu_dying_mask)
extern atomic_t __num_online_cpus;
+extern atomic_t __num_possible_cpus;
extern cpumask_t cpus_booted_once_mask;
@@ -870,19 +871,8 @@ void init_cpu_present(const struct cpumask *src);
void init_cpu_possible(const struct cpumask *src);
void init_cpu_online(const struct cpumask *src);
-static inline void reset_cpu_possible_mask(void)
-{
- bitmap_zero(cpumask_bits(&__cpu_possible_mask), NR_CPUS);
-}
-
-static inline void
-set_cpu_possible(unsigned int cpu, bool possible)
-{
- if (possible)
- cpumask_set_cpu(cpu, &__cpu_possible_mask);
- else
- cpumask_clear_cpu(cpu, &__cpu_possible_mask);
-}
+void set_cpu_possible(unsigned int cpu, bool possible);
+void reset_cpu_possible_mask(void);
static inline void
set_cpu_present(unsigned int cpu, bool present)
@@ -962,7 +952,19 @@ static inline unsigned int num_online_cpus(void)
{
return atomic_read(&__num_online_cpus);
}
-#define num_possible_cpus() cpumask_weight(cpu_possible_mask)
+
+/**
+ * num_possible_cpus() - Read the number of possible CPUs
+ *
+ * Despite the fact that __num_possible_cpus is of type atomic_t, this
+ * interface gives only a momentary snapshot and is not protected against
+ * concurrent CPU hotplug operations unless invoked from a cpuhp_lock held
+ * region.
+ */
+static inline unsigned int num_possible_cpus(void)
+{
+ return atomic_read(&__num_possible_cpus);
+}
#define num_present_cpus() cpumask_weight(cpu_present_mask)
#define num_active_cpus() cpumask_weight(cpu_active_mask)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index cd7605204d4d..a0a815911173 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -2583,10 +2583,13 @@ EXPORT_SYMBOL(cpu_all_bits);
#ifdef CONFIG_INIT_ALL_POSSIBLE
struct cpumask __cpu_possible_mask __read_mostly
= {CPU_BITS_ALL};
+atomic_t __num_possible_cpus __read_mostly = ATOMIC_INIT(NR_CPUS);
#else
struct cpumask __cpu_possible_mask __read_mostly;
+atomic_t __num_possible_cpus __read_mostly;
#endif
EXPORT_SYMBOL(__cpu_possible_mask);
+EXPORT_SYMBOL(__num_possible_cpus);
struct cpumask __cpu_online_mask __read_mostly;
EXPORT_SYMBOL(__cpu_online_mask);
@@ -2611,6 +2614,7 @@ void init_cpu_present(const struct cpumask *src)
void init_cpu_possible(const struct cpumask *src)
{
cpumask_copy(&__cpu_possible_mask, src);
+ atomic_set(&__num_possible_cpus, cpumask_weight(cpu_possible_mask));
}
void init_cpu_online(const struct cpumask *src)
@@ -2640,6 +2644,24 @@ void set_cpu_online(unsigned int cpu, bool online)
}
}
+void reset_cpu_possible_mask(void)
+{
+ bitmap_zero(cpumask_bits(&__cpu_possible_mask), NR_CPUS);
+ atomic_set(&__num_possible_cpus, 0);
+}
+
+void set_cpu_possible(unsigned int cpu, bool possible)
+{
+ if (possible) {
+ if (!cpumask_test_and_set_cpu(cpu, &__cpu_possible_mask))
+ atomic_inc(&__num_possible_cpus);
+ } else {
+ if (cpumask_test_and_clear_cpu(cpu, &__cpu_possible_mask))
+ atomic_dec(&__num_possible_cpus);
+ }
+}
+EXPORT_SYMBOL(set_cpu_possible);
+
/*
* Activate the first processor.
*/
--
2.30.2
^ permalink raw reply related
* [PATCH 14/17] kernel/cpu: add num_present_cpu counter
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Similarly to the online cpus, the cpu_present_mask is actively used
in the kernel. This patch adds a counter for present cpus, so that
users that call num_present_cpus() would know the result immediately,
instead of calling the bitmap_weight for the mask.
Suggested-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
include/linux/cpumask.h | 25 +++++++++++++++----------
kernel/cpu.c | 16 ++++++++++++++++
2 files changed, 31 insertions(+), 10 deletions(-)
diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h
index 0be2504d8e4c..c2a9d15e2cbd 100644
--- a/include/linux/cpumask.h
+++ b/include/linux/cpumask.h
@@ -100,6 +100,7 @@ extern struct cpumask __cpu_dying_mask;
extern atomic_t __num_online_cpus;
extern atomic_t __num_possible_cpus;
+extern atomic_t __num_present_cpus;
extern cpumask_t cpus_booted_once_mask;
@@ -873,15 +874,7 @@ void init_cpu_online(const struct cpumask *src);
void set_cpu_possible(unsigned int cpu, bool possible);
void reset_cpu_possible_mask(void);
-
-static inline void
-set_cpu_present(unsigned int cpu, bool present)
-{
- if (present)
- cpumask_set_cpu(cpu, &__cpu_present_mask);
- else
- cpumask_clear_cpu(cpu, &__cpu_present_mask);
-}
+void set_cpu_present(unsigned int cpu, bool present);
void set_cpu_online(unsigned int cpu, bool online);
@@ -965,7 +958,19 @@ static inline unsigned int num_possible_cpus(void)
{
return atomic_read(&__num_possible_cpus);
}
-#define num_present_cpus() cpumask_weight(cpu_present_mask)
+
+/**
+ * num_present_cpus() - Read the number of present CPUs
+ *
+ * Despite the fact that __num_present_cpus is of type atomic_t, this
+ * interface gives only a momentary snapshot and is not protected against
+ * concurrent CPU hotplug operations unless invoked from a cpuhp_lock held
+ * region.
+ */
+static inline unsigned int num_present_cpus(void)
+{
+ return atomic_read(&__num_present_cpus);
+}
#define num_active_cpus() cpumask_weight(cpu_active_mask)
static inline bool cpu_online(unsigned int cpu)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index a0a815911173..1f7ea1bdde1a 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -2597,6 +2597,9 @@ EXPORT_SYMBOL(__cpu_online_mask);
struct cpumask __cpu_present_mask __read_mostly;
EXPORT_SYMBOL(__cpu_present_mask);
+atomic_t __num_present_cpus __read_mostly;
+EXPORT_SYMBOL(__num_present_cpus);
+
struct cpumask __cpu_active_mask __read_mostly;
EXPORT_SYMBOL(__cpu_active_mask);
@@ -2609,6 +2612,7 @@ EXPORT_SYMBOL(__num_online_cpus);
void init_cpu_present(const struct cpumask *src)
{
cpumask_copy(&__cpu_present_mask, src);
+ atomic_set(&__num_present_cpus, cpumask_weight(cpu_present_mask));
}
void init_cpu_possible(const struct cpumask *src)
@@ -2662,6 +2666,18 @@ void set_cpu_possible(unsigned int cpu, bool possible)
}
EXPORT_SYMBOL(set_cpu_possible);
+void set_cpu_present(unsigned int cpu, bool present)
+{
+ if (present) {
+ if (!cpumask_test_and_set_cpu(cpu, &__cpu_present_mask))
+ atomic_inc(&__num_present_cpus);
+ } else {
+ if (cpumask_test_and_clear_cpu(cpu, &__cpu_present_mask))
+ atomic_dec(&__num_present_cpus);
+ }
+}
+EXPORT_SYMBOL(set_cpu_present);
+
/*
* Activate the first processor.
*/
--
2.30.2
^ permalink raw reply related
* [PATCH 15/17] kernel/cpu: add num_active_cpu counter
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Similarly to the online cpus, the cpu_active_mask is actively used
in the kernel. This patch adds a counter for active cpus, so that
users that call num_active_cpus() would know the result immediately,
instead of calling the bitmap_weight for the mask.
Suggested-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
include/linux/cpumask.h | 26 +++++++++++++++-----------
kernel/cpu.c | 15 +++++++++++++++
2 files changed, 30 insertions(+), 11 deletions(-)
diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h
index c2a9d15e2cbd..0add872898f8 100644
--- a/include/linux/cpumask.h
+++ b/include/linux/cpumask.h
@@ -101,6 +101,7 @@ extern struct cpumask __cpu_dying_mask;
extern atomic_t __num_online_cpus;
extern atomic_t __num_possible_cpus;
extern atomic_t __num_present_cpus;
+extern atomic_t __num_active_cpus;
extern cpumask_t cpus_booted_once_mask;
@@ -875,17 +876,8 @@ void init_cpu_online(const struct cpumask *src);
void set_cpu_possible(unsigned int cpu, bool possible);
void reset_cpu_possible_mask(void);
void set_cpu_present(unsigned int cpu, bool present);
-
void set_cpu_online(unsigned int cpu, bool online);
-
-static inline void
-set_cpu_active(unsigned int cpu, bool active)
-{
- if (active)
- cpumask_set_cpu(cpu, &__cpu_active_mask);
- else
- cpumask_clear_cpu(cpu, &__cpu_active_mask);
-}
+void set_cpu_active(unsigned int cpu, bool active);
static inline void
set_cpu_dying(unsigned int cpu, bool dying)
@@ -971,7 +963,19 @@ static inline unsigned int num_present_cpus(void)
{
return atomic_read(&__num_present_cpus);
}
-#define num_active_cpus() cpumask_weight(cpu_active_mask)
+
+/**
+ * num_active_cpus() - Read the number of active CPUs
+ *
+ * Despite the fact that __num_active_cpus is of type atomic_t, this
+ * interface gives only a momentary snapshot and is not protected against
+ * concurrent CPU hotplug operations unless invoked from a cpuhp_lock held
+ * region.
+ */
+static inline unsigned int num_active_cpus(void)
+{
+ return atomic_read(&__num_active_cpus);
+}
static inline bool cpu_online(unsigned int cpu)
{
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 1f7ea1bdde1a..62b411d88810 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -2603,6 +2603,9 @@ EXPORT_SYMBOL(__num_present_cpus);
struct cpumask __cpu_active_mask __read_mostly;
EXPORT_SYMBOL(__cpu_active_mask);
+atomic_t __num_active_cpus __read_mostly;
+EXPORT_SYMBOL(__num_active_cpus);
+
struct cpumask __cpu_dying_mask __read_mostly;
EXPORT_SYMBOL(__cpu_dying_mask);
@@ -2678,6 +2681,18 @@ void set_cpu_present(unsigned int cpu, bool present)
}
EXPORT_SYMBOL(set_cpu_present);
+void set_cpu_active(unsigned int cpu, bool active)
+{
+ if (active) {
+ if (!cpumask_test_and_set_cpu(cpu, &__cpu_active_mask))
+ atomic_inc(&__num_active_cpus);
+ } else {
+ if (cpumask_test_and_clear_cpu(cpu, &__cpu_active_mask))
+ atomic_dec(&__num_active_cpus);
+ }
+}
+EXPORT_SYMBOL(set_cpu_active);
+
/*
* Activate the first processor.
*/
--
2.30.2
^ permalink raw reply related
* [PATCH 16/17] tools/bitmap: sync bitmap_weight
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
Pull bitmap_weight_{cmp,eq,gt,ge,lt,le} from mother kernel and
use where applicable.
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
tools/include/linux/bitmap.h | 44 ++++++++++++++++++++++++++++++++++++
tools/lib/bitmap.c | 20 ++++++++++++++++
tools/perf/util/pmu.c | 2 +-
3 files changed, 65 insertions(+), 1 deletion(-)
diff --git a/tools/include/linux/bitmap.h b/tools/include/linux/bitmap.h
index ea97804d04d4..e8ae9a85d555 100644
--- a/tools/include/linux/bitmap.h
+++ b/tools/include/linux/bitmap.h
@@ -12,6 +12,8 @@
unsigned long name[BITS_TO_LONGS(bits)]
int __bitmap_weight(const unsigned long *bitmap, int bits);
+int __bitmap_weight_cmp(const unsigned long *bitmap, unsigned int bits,
+ unsigned int num);
void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits);
int __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
@@ -68,6 +70,48 @@ static inline int bitmap_weight(const unsigned long *src, unsigned int nbits)
return __bitmap_weight(src, nbits);
}
+static __always_inline
+int bitmap_weight_cmp(const unsigned long *src, unsigned int nbits, int num)
+{
+ if (num > (int)nbits || num < 0)
+ return -num;
+
+ if (small_const_nbits(nbits))
+ return hweight_long(*src & BITMAP_LAST_WORD_MASK(nbits)) - num;
+
+ return __bitmap_weight_cmp(src, nbits, num);
+}
+
+static __always_inline
+bool bitmap_weight_eq(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num) == 0;
+}
+
+static __always_inline
+bool bitmap_weight_gt(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num) > 0;
+}
+
+static __always_inline
+bool bitmap_weight_ge(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num - 1) > 0;
+}
+
+static __always_inline
+bool bitmap_weight_lt(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num - 1) <= 0;
+}
+
+static __always_inline
+bool bitmap_weight_le(const unsigned long *src, unsigned int nbits, int num)
+{
+ return bitmap_weight_cmp(src, nbits, num) <= 0;
+}
+
static inline void bitmap_or(unsigned long *dst, const unsigned long *src1,
const unsigned long *src2, unsigned int nbits)
{
diff --git a/tools/lib/bitmap.c b/tools/lib/bitmap.c
index db466ef7be9d..06e58fee8523 100644
--- a/tools/lib/bitmap.c
+++ b/tools/lib/bitmap.c
@@ -18,6 +18,26 @@ int __bitmap_weight(const unsigned long *bitmap, int bits)
return w;
}
+int __bitmap_weight_cmp(const unsigned long *bitmap, unsigned int bits, int num)
+{
+ unsigned int k, w, lim = bits / BITS_PER_LONG;
+
+ for (k = 0, w = 0; k < lim; k++) {
+ if (w + bits - k * BITS_PER_LONG < num)
+ goto out;
+
+ w += hweight_long(bitmap[k]);
+
+ if (w > num)
+ goto out;
+ }
+
+ if (bits % BITS_PER_LONG)
+ w += hweight_long(bitmap[k] & BITMAP_LAST_WORD_MASK(bits));
+out:
+ return w - num;
+}
+
void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
index 6ae58406f4fc..015ee1321c7c 100644
--- a/tools/perf/util/pmu.c
+++ b/tools/perf/util/pmu.c
@@ -1314,7 +1314,7 @@ static int pmu_config_term(const char *pmu_name,
*/
if (term->type_val == PARSE_EVENTS__TERM_TYPE_NUM) {
if (term->no_value &&
- bitmap_weight(format->bits, PERF_PMU_FORMAT_BITS) > 1) {
+ bitmap_weight_gt(format->bits, PERF_PMU_FORMAT_BITS, 1)) {
if (err) {
parse_events_error__handle(err, term->err_val,
strdup("no value assigned for term"),
--
2.30.2
^ permalink raw reply related
* [PATCH 17/17] MAINTAINERS: add cpumask and nodemask files to BITMAP_API
From: Yury Norov @ 2021-12-18 21:20 UTC (permalink / raw)
To: linux-kernel, Yury Norov, James E.J. Bottomley,
Martin K. Petersen, Michał Mirosław, Paul E. McKenney,
Rafael J. Wysocki, Alexander Shishkin, Alexey Klimov,
Amitkumar Karwar, Andi Kleen, Andrew Lunn, Andrew Morton,
Andy Gross, Andy Lutomirski, Andy Shevchenko, Anup Patel,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Arnd Bergmann,
Borislav Petkov, Catalin Marinas, Christoph Hellwig,
Christoph Lameter, Daniel Vetter, Dave Hansen, David Airlie,
David Laight, Dennis Zhou, Emil Renner Berthing,
Geert Uytterhoeven, Geetha sowjanya, Greg Kroah-Hartman, Guo Ren,
Hans de Goede, Heiko Carstens, Ian Rogers, Ingo Molnar,
Jakub Kicinski, Jason Wessel, Jens Axboe, Jiri Olsa, Joe Perches,
Jonathan Cameron, Juri Lelli, Kees Cook, Krzysztof Kozlowski,
Lee Jones, Marc Zyngier, Marcin Wojtas, Mark Gross, Mark Rutland,
Matti Vaittinen, Mauro Carvalho Chehab, Mel Gorman,
Michael Ellerman, Mike Marciniszyn, Nicholas Piggin,
Palmer Dabbelt, Peter Zijlstra, Petr Mladek, Randy Dunlap,
Rasmus Villemoes, Russell King, Saeed Mahameed, Sagi Grimberg,
Sergey Senozhatsky, Solomon Peachy, Stephen Boyd,
Stephen Rothwell, Steven Rostedt, Subbaraya Sundeep, Sudeep Holla,
Sunil Goutham, Tariq Toukan, Tejun Heo, Thomas Bogendoerfer,
Thomas Gleixner, Ulf Hansson, Vincent Guittot, Vineet Gupta,
Viresh Kumar, Vivien Didelot, Vlastimil Babka, Will Deacon,
bcm-kernel-feedback-list, kvm, linux-alpha, linux-arm-kernel,
linux-crypto, linux-csky, linux-ia64, linux-mips, linux-mm,
linux-perf-users, linux-riscv, linux-s390, linux-snps-arc,
linuxppc-dev
In-Reply-To: <20211218212014.1315894-1-yury.norov@gmail.com>
cpumask and nodemask APIs are thin wrappers around basic bitmap API, and
corresponding files are not formally maintained. This patch adds them to
BITMAP_API section, so that bitmap folks would have closer look at it.
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
MAINTAINERS | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 5964e047bc04..ecd41988c871 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3392,10 +3392,14 @@ R: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
R: Rasmus Villemoes <linux@rasmusvillemoes.dk>
S: Maintained
F: include/linux/bitmap.h
+F: include/linux/cpumask.h
F: include/linux/find.h
+F: include/linux/nodemask.h
F: lib/bitmap.c
+F: lib/cpumask.c
F: lib/find_bit.c
F: lib/find_bit_benchmark.c
+F: lib/nodemask.c
F: lib/test_bitmap.c
F: tools/include/linux/bitmap.h
F: tools/include/linux/find.h
--
2.30.2
^ permalink raw reply related
* Re: [PATCH 01/17] all: don't use bitmap_weight() where possible
From: Michał Mirosław @ 2021-12-18 22:15 UTC (permalink / raw)
To: Yury Norov
Cc: Juri Lelli, Andrew Lunn, Rafael J. Wysocki, Catalin Marinas,
Guo Ren, Christoph Lameter, Christoph Hellwig, Andi Kleen,
Vincent Guittot, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
Viresh Kumar, Petr Mladek, Arnaldo Carvalho de Melo, Jens Axboe,
Andy Lutomirski, Lee Jones, Greg Kroah-Hartman, Randy Dunlap,
linux-kernel, linux-perf-users, Sergey Senozhatsky,
Thomas Gleixner, linux-crypto, Joe Perches, Andrew Morton,
Mark Rutland, Anup Patel, linux-ia64, David Airlie, Dave Hansen,
Solomon Peachy, Stephen Rothwell, Krzysztof Kozlowski,
Dennis Zhou, Matti Vaittinen, Sudeep Holla, Tejun Heo,
linux-arm-kernel, Stephen Boyd, Tariq Toukan, Jonathan Cameron,
Ulf Hansson, Emil Renner Berthing, Alexander Shishkin,
Mike Marciniszyn, Rasmus Villemoes, Subbaraya Sundeep,
Will Deacon, Sagi Grimberg, linux-csky, bcm-kernel-feedback-list,
linux-snps-arc, Kees Cook, Arnd Bergmann, James E.J. Bottomley,
Vineet Gupta, Steven Rostedt, Mark Gross, Borislav Petkov,
Mauro Carvalho Chehab, Thomas Bogendoerfer, Martin K. Petersen,
David Laight, linux-alpha, Geetha sowjanya, Ian Rogers, kvm,
Peter Zijlstra, Amitkumar Karwar, linux-mm, linux-riscv,
Jiri Olsa, Ard Biesheuvel, Marc Zyngier, Russell King, Andy Gross,
Jakub Kicinski, Vivien Didelot, Sunil Goutham, Paul E. McKenney,
linux-s390, Alexey Klimov, Heiko Carstens, Hans de Goede,
Nicholas Piggin, Marcin Wojtas, Vlastimil Babka, linuxppc-dev,
linux-mips, Palmer Dabbelt, Daniel Vetter, Jason Wessel,
Saeed Mahameed, Andy Shevchenko
In-Reply-To: <20211218212014.1315894-2-yury.norov@gmail.com>
On Sat, Dec 18, 2021 at 01:19:57PM -0800, Yury Norov wrote:
> Don't call bitmap_weight() if the following code can get by
> without it.
>
> Signed-off-by: Yury Norov <yury.norov@gmail.com>
> ---
> drivers/net/dsa/b53/b53_common.c | 6 +-----
> drivers/net/ethernet/broadcom/bcmsysport.c | 6 +-----
> drivers/thermal/intel/intel_powerclamp.c | 9 +++------
> 3 files changed, 5 insertions(+), 16 deletions(-)
[...]
Looks good, but I think this needs to be split per subsystem.
Best Regards
Michał Mirosław
^ permalink raw reply
* Re: [PATCH 01/17] all: don't use bitmap_weight() where possible
From: Yury Norov @ 2021-12-18 23:28 UTC (permalink / raw)
To: Michał Mirosław
Cc: Juri Lelli, Andrew Lunn, Rafael J. Wysocki, Catalin Marinas,
Guo Ren, Christoph Lameter, Christoph Hellwig, Andi Kleen,
Vincent Guittot, Ingo Molnar, Geert Uytterhoeven, Mel Gorman,
Viresh Kumar, Petr Mladek, Arnaldo Carvalho de Melo, Jens Axboe,
Andy Lutomirski, Lee Jones, Greg Kroah-Hartman, Randy Dunlap,
linux-kernel, linux-perf-users, Sergey Senozhatsky,
Thomas Gleixner, linux-crypto, Joe Perches, Andrew Morton,
Mark Rutland, Anup Patel, linux-ia64, David Airlie, Dave Hansen,
Solomon Peachy, Stephen Rothwell, Krzysztof Kozlowski,
Dennis Zhou, Matti Vaittinen, Sudeep Holla, Tejun Heo,
linux-arm-kernel, Stephen Boyd, Tariq Toukan, Jonathan Cameron,
Ulf Hansson, Emil Renner Berthing, Alexander Shishkin,
Mike Marciniszyn, Rasmus Villemoes, Subbaraya Sundeep,
Will Deacon, Sagi Grimberg, linux-csky, bcm-kernel-feedback-list,
linux-snps-arc, Kees Cook, Arnd Bergmann, James E.J. Bottomley,
Vineet Gupta, Steven Rostedt, Mark Gross, Borislav Petkov,
Mauro Carvalho Chehab, Thomas Bogendoerfer, Martin K. Petersen,
David Laight, linux-alpha, Geetha sowjanya, Ian Rogers, kvm,
Peter Zijlstra, Amitkumar Karwar, linux-mm, linux-riscv,
Jiri Olsa, Ard Biesheuvel, Marc Zyngier, Russell King, Andy Gross,
Jakub Kicinski, Vivien Didelot, Sunil Goutham, Paul E. McKenney,
linux-s390, Alexey Klimov, Heiko Carstens, Hans de Goede,
Nicholas Piggin, Marcin Wojtas, Vlastimil Babka, linuxppc-dev,
linux-mips, Palmer Dabbelt, Daniel Vetter, Jason Wessel,
Saeed Mahameed, Andy Shevchenko
In-Reply-To: <Yb5dmqlYd3owtH29@qmqm.qmqm.pl>
On Sat, Dec 18, 2021 at 2:16 PM Michał Mirosław <mirq-linux@rere.qmqm.pl> wrote:
>
> On Sat, Dec 18, 2021 at 01:19:57PM -0800, Yury Norov wrote:
> > Don't call bitmap_weight() if the following code can get by
> > without it.
> >
> > Signed-off-by: Yury Norov <yury.norov@gmail.com>
> > ---
> > drivers/net/dsa/b53/b53_common.c | 6 +-----
> > drivers/net/ethernet/broadcom/bcmsysport.c | 6 +-----
> > drivers/thermal/intel/intel_powerclamp.c | 9 +++------
> > 3 files changed, 5 insertions(+), 16 deletions(-)
> [...]
>
> Looks good,
Does it mean Acked-by, Reviewed-by, or something else?
> but I think this needs to be split per subsystem.
What you ask breaks rules:
Documentation/process/submitting-patches.rst:
Separate each **logical change** into a separate patch.
For example, if your changes include both bug fixes and performance
enhancements for a single driver, separate those changes into two
or more patches. If your changes include an API update, and a new
driver which uses that new API, separate those into two patches.
On the other hand, if you make a single change to numerous files,
group those changes into a single patch. Thus a single logical change
is contained within a single patch.
This is not a dead rule, refer for example the 96d4f267e40f9 ("Remove
'type' argument from access_ok() functioin.")
Or this: https://lkml.org/lkml/2021/6/14/1736
Thanks,
Yury
^ permalink raw reply
* Re: [PATCH/RFC] mm: add and use batched version of __tlb_remove_table()
From: Dave Hansen @ 2021-12-19 1:34 UTC (permalink / raw)
To: Nikita Yushchenko, Will Deacon, Aneesh Kumar K.V, Andrew Morton,
Nick Piggin, Peter Zijlstra, Catalin Marinas, Heiko Carstens,
Vasily Gorbik, Christian Borntraeger, David S. Miller,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
Arnd Bergmann
Cc: linux-arch, linux-s390, x86, linux-kernel, linux-mm, kernel,
sparclinux, linuxppc-dev
In-Reply-To: <d6094dc4-3976-e06f-696b-c55f696fe287@virtuozzo.com>
On 12/18/21 6:31 AM, Nikita Yushchenko wrote:
>>> This allows archs to optimize it, by
>>> freeing multiple tables in a single release_pages() call. This is
>>> faster than individual put_page() calls, especially with memcg
>>> accounting enabled.
>>
>> Could we quantify "faster"? There's a non-trivial amount of code being
>> added here and it would be nice to back it up with some cold-hard
>> numbers.
>
> I currently don't have numbers for this patch taken alone. This patch
> originates from work done some years ago to reduce cost of memory
> accounting, and x86-only version of this patch was in virtuozzo/openvz
> kernel since then. Other patches from that work have been upstreamed,
> but this one was missed.
>
> Still it's obvious that release_pages() shall be faster that a loop
> calling put_page() - isn't that exactly the reason why release_pages()
> exists and is different from a loop calling put_page()?
Yep, but this patch does a bunch of stuff to some really hot paths. It
would be greatly appreciated if you could put in the effort to actually
put some numbers behind this. Plenty of weird stuff happens on
computers that we suck at predicting.
I'd be happy with even a quick little micro. My favorite is:
https://github.com/antonblanchard/will-it-scale
Although, I do wonder if anything will even be measurable. Please at
least try.
...
>> But, even more than that, do all the architectures even need the
>> free_swap_cache()?
>
> I was under impression that process page tables are a valid target for
> swapping out. Although I can be wrong here.
It's not out of the realm of possibilities. But, last I checked, the
only path we free page tables in was when VMAs are being torn down. I
have a longstanding TODO item to reclaim them if they're empty (all
zeros) or to zero them out if they're mapping page cache.
^ permalink raw reply
* [GIT PULL] Please pull powerpc/linux.git powerpc-5.16-4 tag
From: Michael Ellerman @ 2021-12-19 3:53 UTC (permalink / raw)
To: Linus Torvalds; +Cc: nixiaoming, linuxppc-dev, linux-kernel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi Linus,
Please pull some more powerpc fixes for 5.16:
The following changes since commit 5bb60ea611db1e04814426ed4bd1c95d1487678e:
powerpc/32: Fix hardlockup on vmap stack overflow (2021-11-24 21:00:51 +1100)
are available in the git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git tags/powerpc-5.16-4
for you to fetch changes up to 8734b41b3efe0fc6082c1937b0e88556c396dc96:
powerpc/module_64: Fix livepatching for RO modules (2021-12-14 23:13:03 +1100)
- ------------------------------------------------------------------
powerpc fixes for 5.16 #4
Fix a recently introduced oops at boot on 85xx in some configurations.
Fix crashes when loading some livepatch modules with STRICT_MODULE_RWX.
Thanks to: Joe Lawrence, Russell Currey, Xiaoming Ni.
- ------------------------------------------------------------------
Russell Currey (1):
powerpc/module_64: Fix livepatching for RO modules
Xiaoming Ni (1):
powerpc/85xx: Fix oops when CONFIG_FSL_PMC=n
arch/powerpc/kernel/module_64.c | 42 ++++++++++++++++----
arch/powerpc/platforms/85xx/smp.c | 4 +-
2 files changed, 36 insertions(+), 10 deletions(-)
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEJFGtCPCthwEv2Y/bUevqPMjhpYAFAmG+rMQACgkQUevqPMjh
pYBLNQ/8DdqJFMgeVNV+RBinTdrtXl4RmMGvNE0ogmZ215GPiwjbjt2WYlMEg0Ol
4WrWSZwThKJdDXvR05HTU9QncNEXi1KIISfppNkNJQkRzRzs7HjDgFMjinNBsLzg
q3Q8LeGhj2eL3VNbavAr5UBsXgxJriGjrQQQj6NR2jyLZnITd+Wdn4/SWUgxdlmn
+aSKvoOWAUM3/+SrrhDmCsCKhM24bwoQ339uOp81j63oP6UVtr4dFVvDcVZyL+ea
tYSIQ5UeO8/zoaEMudZwFRifXmPk6+KiYzlzY+bx/6b5xXTv+B1oAT0RX+nvdv2g
aQIRI0EKsEf2OLD8ttlGKkF8NyXQBJjsDZ6VBklXTcSlIxMtSAJxz80USHk2FyHx
ojgi51Axdp031gA7bhq6jDvSO1izYR4S7si3Hqfzm4IxYMr5UnQSA5IJfsfMlASt
3qZBFpkN3rwFOERMa9IiLarnLzMiionvw7Vf2nESx+Uv/0sYS38528Z2xjBgagdd
OUA5h8XsC3NLUpbZtQy9R0+7jkDYXbe0r5BoPjmonkS90bR3/LhsO8eCWANqsaWO
2Jtu6V33PgAKZbR02V2o76t0WSzf3z72kOq7vsvcmc4tD6Lv4ph3d2ynjQ9oHizu
7FQqoFdHAA/Cbq/AiZqIVwE1pUpJIKesjr/JcNkagUsExLTnXQk=
=Xy0P
-----END PGP SIGNATURE-----
^ permalink raw reply
* Re: [GIT PULL] Please pull powerpc/linux.git powerpc-5.16-4 tag
From: pr-tracker-bot @ 2021-12-19 20:34 UTC (permalink / raw)
To: Michael Ellerman; +Cc: nixiaoming, Linus Torvalds, linuxppc-dev, linux-kernel
In-Reply-To: <87lf0hl06g.fsf@mpe.ellerman.id.au>
The pull request you sent on Sun, 19 Dec 2021 14:53:59 +1100:
> https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git tags/powerpc-5.16-4
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/713ab911f2cd5ef00b5072aacd84bd93a444ae12
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply
* [PATCH kernel v4] KVM: PPC: Merge powerpc's debugfs entry content into generic entry
From: Alexey Kardashevskiy @ 2021-12-20 1:23 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Fabiano Rosas, Alexey Kardashevskiy, Nicholas Piggin,
kvm-ppc, Cédric Le Goater, David Stevens
At the moment KVM on PPC creates 4 types of entries under the kvm debugfs:
1) "%pid-%fd" per a KVM instance (for all platforms);
2) "vm%pid" (for PPC Book3s HV KVM);
3) "vm%u_vcpu%u_timing" (for PPC Book3e KVM);
4) "kvm-xive-%p" (for XIVE PPC Book3s KVM, the same for XICS);
The problem with this is that multiple VMs per process is not allowed for
2) and 3) which makes it possible for userspace to trigger errors when
creating duplicated debugfs entries.
This merges all these into 1).
This defines kvm_arch_create_kvm_debugfs() similar to
kvm_arch_create_vcpu_debugfs().
This defines 2 hooks in kvmppc_ops that allow specific KVM implementations
add necessary entries, this adds the _e500 suffix to
kvmppc_create_vcpu_debugfs_e500() to make it clear what platform it is for.
This makes use of already existing kvm_arch_create_vcpu_debugfs() on PPC.
This removes no more used debugfs_dir pointers from PPC kvm_arch structs.
This stops removing vcpu entries as once created vcpus stay around
for the entire life of a VM and removed when the KVM instance is closed,
see commit d56f5136b010 ("KVM: let kvm_destroy_vm_debugfs clean up vCPU
debugfs directories").
Suggested-by: Fabiano Rosas <farosas@linux.ibm.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
v4:
* added "kvm-xive-%p"
v3:
* reworked commit log, especially, the bit about removing vcpus
v2:
* handled powerpc-booke
* s/kvm/vm/ in arch hooks
---
arch/powerpc/include/asm/kvm_host.h | 6 ++---
arch/powerpc/include/asm/kvm_ppc.h | 2 ++
arch/powerpc/kvm/timing.h | 9 ++++----
arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +-
arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +-
arch/powerpc/kvm/book3s_hv.c | 31 ++++++++++----------------
arch/powerpc/kvm/book3s_xics.c | 13 ++---------
arch/powerpc/kvm/book3s_xive.c | 13 ++---------
arch/powerpc/kvm/book3s_xive_native.c | 13 ++---------
arch/powerpc/kvm/e500.c | 1 +
arch/powerpc/kvm/e500mc.c | 1 +
arch/powerpc/kvm/powerpc.c | 16 ++++++++++---
arch/powerpc/kvm/timing.c | 20 ++++-------------
13 files changed, 47 insertions(+), 82 deletions(-)
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 17263276189e..f5e14fa683f4 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -26,6 +26,8 @@
#include <asm/hvcall.h>
#include <asm/mce.h>
+#define __KVM_HAVE_ARCH_VCPU_DEBUGFS
+
#define KVM_MAX_VCPUS NR_CPUS
#define KVM_MAX_VCORES NR_CPUS
@@ -295,7 +297,6 @@ struct kvm_arch {
bool dawr1_enabled;
pgd_t *pgtable;
u64 process_table;
- struct dentry *debugfs_dir;
struct kvm_resize_hpt *resize_hpt; /* protected by kvm->lock */
#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */
#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
@@ -673,7 +674,6 @@ struct kvm_vcpu_arch {
u64 timing_min_duration[__NUMBER_OF_KVM_EXIT_TYPES];
u64 timing_max_duration[__NUMBER_OF_KVM_EXIT_TYPES];
u64 timing_last_exit;
- struct dentry *debugfs_exit_timing;
#endif
#ifdef CONFIG_PPC_BOOK3S
@@ -829,8 +829,6 @@ struct kvm_vcpu_arch {
struct kvmhv_tb_accumulator rm_exit; /* real-mode exit code */
struct kvmhv_tb_accumulator guest_time; /* guest execution */
struct kvmhv_tb_accumulator cede_time; /* time napping inside guest */
-
- struct dentry *debugfs_dir;
#endif /* CONFIG_KVM_BOOK3S_HV_EXIT_TIMING */
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 33db83b82fbd..d2b192dea0d2 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -316,6 +316,8 @@ struct kvmppc_ops {
int (*svm_off)(struct kvm *kvm);
int (*enable_dawr1)(struct kvm *kvm);
bool (*hash_v3_possible)(void);
+ int (*create_vm_debugfs)(struct kvm *kvm);
+ int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry);
};
extern struct kvmppc_ops *kvmppc_hv_ops;
diff --git a/arch/powerpc/kvm/timing.h b/arch/powerpc/kvm/timing.h
index feef7885ba82..493a7d510fd5 100644
--- a/arch/powerpc/kvm/timing.h
+++ b/arch/powerpc/kvm/timing.h
@@ -14,8 +14,8 @@
#ifdef CONFIG_KVM_EXIT_TIMING
void kvmppc_init_timing_stats(struct kvm_vcpu *vcpu);
void kvmppc_update_timing_stats(struct kvm_vcpu *vcpu);
-void kvmppc_create_vcpu_debugfs(struct kvm_vcpu *vcpu, unsigned int id);
-void kvmppc_remove_vcpu_debugfs(struct kvm_vcpu *vcpu);
+void kvmppc_create_vcpu_debugfs_e500(struct kvm_vcpu *vcpu,
+ struct dentry *debugfs_dentry);
static inline void kvmppc_set_exit_type(struct kvm_vcpu *vcpu, int type)
{
@@ -26,9 +26,8 @@ static inline void kvmppc_set_exit_type(struct kvm_vcpu *vcpu, int type)
/* if exit timing is not configured there is no need to build the c file */
static inline void kvmppc_init_timing_stats(struct kvm_vcpu *vcpu) {}
static inline void kvmppc_update_timing_stats(struct kvm_vcpu *vcpu) {}
-static inline void kvmppc_create_vcpu_debugfs(struct kvm_vcpu *vcpu,
- unsigned int id) {}
-static inline void kvmppc_remove_vcpu_debugfs(struct kvm_vcpu *vcpu) {}
+static inline void kvmppc_create_vcpu_debugfs_e500(struct kvm_vcpu *vcpu,
+ struct dentry *debugfs_dentry) {}
static inline void kvmppc_set_exit_type(struct kvm_vcpu *vcpu, int type) {}
#endif /* CONFIG_KVM_EXIT_TIMING */
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index c63e263312a4..33dae253a0ac 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -2112,7 +2112,7 @@ static const struct file_operations debugfs_htab_fops = {
void kvmppc_mmu_debugfs_init(struct kvm *kvm)
{
- debugfs_create_file("htab", 0400, kvm->arch.debugfs_dir, kvm,
+ debugfs_create_file("htab", 0400, kvm->debugfs_dentry, kvm,
&debugfs_htab_fops);
}
diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c
index 8cebe5542256..e4ce2a35483f 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c
@@ -1454,7 +1454,7 @@ static const struct file_operations debugfs_radix_fops = {
void kvmhv_radix_debugfs_init(struct kvm *kvm)
{
- debugfs_create_file("radix", 0400, kvm->arch.debugfs_dir, kvm,
+ debugfs_create_file("radix", 0400, kvm->debugfs_dentry, kvm,
&debugfs_radix_fops);
}
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index bf1eb1160ae2..4c52541b6f37 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2768,20 +2768,17 @@ static const struct file_operations debugfs_timings_ops = {
};
/* Create a debugfs directory for the vcpu */
-static void debugfs_vcpu_init(struct kvm_vcpu *vcpu, unsigned int id)
+static int kvmppc_arch_create_vcpu_debugfs_hv(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry)
{
- char buf[16];
- struct kvm *kvm = vcpu->kvm;
-
- snprintf(buf, sizeof(buf), "vcpu%u", id);
- vcpu->arch.debugfs_dir = debugfs_create_dir(buf, kvm->arch.debugfs_dir);
- debugfs_create_file("timings", 0444, vcpu->arch.debugfs_dir, vcpu,
+ debugfs_create_file("timings", 0444, debugfs_dentry, vcpu,
&debugfs_timings_ops);
+ return 0;
}
#else /* CONFIG_KVM_BOOK3S_HV_EXIT_TIMING */
-static void debugfs_vcpu_init(struct kvm_vcpu *vcpu, unsigned int id)
+static int kvmppc_arch_create_vcpu_debugfs_hv(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry)
{
+ return 0;
}
#endif /* CONFIG_KVM_BOOK3S_HV_EXIT_TIMING */
@@ -2904,8 +2901,6 @@ static int kvmppc_core_vcpu_create_hv(struct kvm_vcpu *vcpu)
vcpu->arch.cpu_type = KVM_CPU_3S_64;
kvmppc_sanity_check(vcpu);
- debugfs_vcpu_init(vcpu, id);
-
return 0;
}
@@ -5226,7 +5221,6 @@ void kvmppc_free_host_rm_ops(void)
static int kvmppc_core_init_vm_hv(struct kvm *kvm)
{
unsigned long lpcr, lpid;
- char buf[32];
int ret;
mutex_init(&kvm->arch.uvmem_lock);
@@ -5359,15 +5353,14 @@ static int kvmppc_core_init_vm_hv(struct kvm *kvm)
kvm->arch.smt_mode = 1;
kvm->arch.emul_smt_mode = 1;
- /*
- * Create a debugfs directory for the VM
- */
- snprintf(buf, sizeof(buf), "vm%d", current->pid);
- kvm->arch.debugfs_dir = debugfs_create_dir(buf, kvm_debugfs_dir);
+ return 0;
+}
+
+static int kvmppc_arch_create_vm_debugfs_hv(struct kvm *kvm)
+{
kvmppc_mmu_debugfs_init(kvm);
if (radix_enabled())
kvmhv_radix_debugfs_init(kvm);
-
return 0;
}
@@ -5382,8 +5375,6 @@ static void kvmppc_free_vcores(struct kvm *kvm)
static void kvmppc_core_destroy_vm_hv(struct kvm *kvm)
{
- debugfs_remove_recursive(kvm->arch.debugfs_dir);
-
if (!cpu_has_feature(CPU_FTR_ARCH_300))
kvm_hv_vm_deactivated();
@@ -6044,6 +6035,8 @@ static struct kvmppc_ops kvm_ops_hv = {
.svm_off = kvmhv_svm_off,
.enable_dawr1 = kvmhv_enable_dawr1,
.hash_v3_possible = kvmppc_hash_v3_possible,
+ .create_vcpu_debugfs = kvmppc_arch_create_vcpu_debugfs_hv,
+ .create_vm_debugfs = kvmppc_arch_create_vm_debugfs_hv,
};
static int kvm_init_subcore_bitmap(void)
diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c
index ebd5d920de8c..3dfa9285e4a4 100644
--- a/arch/powerpc/kvm/book3s_xics.c
+++ b/arch/powerpc/kvm/book3s_xics.c
@@ -1016,19 +1016,10 @@ DEFINE_SHOW_ATTRIBUTE(xics_debug);
static void xics_debugfs_init(struct kvmppc_xics *xics)
{
- char *name;
-
- name = kasprintf(GFP_KERNEL, "kvm-xics-%p", xics);
- if (!name) {
- pr_err("%s: no memory for name\n", __func__);
- return;
- }
-
- xics->dentry = debugfs_create_file(name, 0444, arch_debugfs_dir,
+ xics->dentry = debugfs_create_file("xics", 0444, xics->kvm->debugfs_dentry,
xics, &xics_debug_fops);
- pr_debug("%s: created %s\n", __func__, name);
- kfree(name);
+ pr_debug("%s: created\n", __func__);
}
static struct kvmppc_ics *kvmppc_xics_create_ics(struct kvm *kvm,
diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c
index 225008882958..bb41afbb68fc 100644
--- a/arch/powerpc/kvm/book3s_xive.c
+++ b/arch/powerpc/kvm/book3s_xive.c
@@ -2351,19 +2351,10 @@ DEFINE_SHOW_ATTRIBUTE(xive_debug);
static void xive_debugfs_init(struct kvmppc_xive *xive)
{
- char *name;
-
- name = kasprintf(GFP_KERNEL, "kvm-xive-%p", xive);
- if (!name) {
- pr_err("%s: no memory for name\n", __func__);
- return;
- }
-
- xive->dentry = debugfs_create_file(name, S_IRUGO, arch_debugfs_dir,
+ xive->dentry = debugfs_create_file("xive", S_IRUGO, xive->kvm->debugfs_dentry,
xive, &xive_debug_fops);
- pr_debug("%s: created %s\n", __func__, name);
- kfree(name);
+ pr_debug("%s: created\n", __func__);
}
static void kvmppc_xive_init(struct kvm_device *dev)
diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c
index 99db9ac49901..e86f5b6c2ae1 100644
--- a/arch/powerpc/kvm/book3s_xive_native.c
+++ b/arch/powerpc/kvm/book3s_xive_native.c
@@ -1259,19 +1259,10 @@ DEFINE_SHOW_ATTRIBUTE(xive_native_debug);
static void xive_native_debugfs_init(struct kvmppc_xive *xive)
{
- char *name;
-
- name = kasprintf(GFP_KERNEL, "kvm-xive-%p", xive);
- if (!name) {
- pr_err("%s: no memory for name\n", __func__);
- return;
- }
-
- xive->dentry = debugfs_create_file(name, 0444, arch_debugfs_dir,
+ xive->dentry = debugfs_create_file("xive", 0444, xive->kvm->debugfs_dentry,
xive, &xive_native_debug_fops);
- pr_debug("%s: created %s\n", __func__, name);
- kfree(name);
+ pr_debug("%s: created\n", __func__);
}
static void kvmppc_xive_native_init(struct kvm_device *dev)
diff --git a/arch/powerpc/kvm/e500.c b/arch/powerpc/kvm/e500.c
index 7e8b69015d20..c8b2b4478545 100644
--- a/arch/powerpc/kvm/e500.c
+++ b/arch/powerpc/kvm/e500.c
@@ -495,6 +495,7 @@ static struct kvmppc_ops kvm_ops_e500 = {
.emulate_op = kvmppc_core_emulate_op_e500,
.emulate_mtspr = kvmppc_core_emulate_mtspr_e500,
.emulate_mfspr = kvmppc_core_emulate_mfspr_e500,
+ .create_vcpu_debugfs = kvmppc_create_vcpu_debugfs_e500,
};
static int __init kvmppc_e500_init(void)
diff --git a/arch/powerpc/kvm/e500mc.c b/arch/powerpc/kvm/e500mc.c
index 1c189b5aadcc..fa0d8dbbe484 100644
--- a/arch/powerpc/kvm/e500mc.c
+++ b/arch/powerpc/kvm/e500mc.c
@@ -381,6 +381,7 @@ static struct kvmppc_ops kvm_ops_e500mc = {
.emulate_op = kvmppc_core_emulate_op_e500,
.emulate_mtspr = kvmppc_core_emulate_mtspr_e500,
.emulate_mfspr = kvmppc_core_emulate_mfspr_e500,
+ .create_vcpu_debugfs = kvmppc_create_vcpu_debugfs_e500,
};
static int __init kvmppc_e500mc_init(void)
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index a72920f4f221..2ea73dfcebb2 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -763,7 +763,6 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
goto out_vcpu_uninit;
vcpu->arch.waitp = &vcpu->wait;
- kvmppc_create_vcpu_debugfs(vcpu, vcpu->vcpu_id);
return 0;
out_vcpu_uninit:
@@ -780,8 +779,6 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
/* Make sure we're not using the vcpu anymore */
hrtimer_cancel(&vcpu->arch.dec_timer);
- kvmppc_remove_vcpu_debugfs(vcpu);
-
switch (vcpu->arch.irq_type) {
case KVMPPC_IRQ_MPIC:
kvmppc_mpic_disconnect_vcpu(vcpu->arch.mpic, vcpu);
@@ -2505,3 +2502,16 @@ int kvm_arch_init(void *opaque)
}
EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_ppc_instr);
+
+void kvm_arch_create_vcpu_debugfs(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry)
+{
+ if (vcpu->kvm->arch.kvm_ops->create_vcpu_debugfs)
+ vcpu->kvm->arch.kvm_ops->create_vcpu_debugfs(vcpu, debugfs_dentry);
+}
+
+int kvm_arch_create_vm_debugfs(struct kvm *kvm)
+{
+ if (kvm->arch.kvm_ops->create_vm_debugfs)
+ kvm->arch.kvm_ops->create_vm_debugfs(kvm);
+ return 0;
+}
diff --git a/arch/powerpc/kvm/timing.c b/arch/powerpc/kvm/timing.c
index ba56a5cbba97..f6d472874c85 100644
--- a/arch/powerpc/kvm/timing.c
+++ b/arch/powerpc/kvm/timing.c
@@ -204,21 +204,9 @@ static const struct file_operations kvmppc_exit_timing_fops = {
.release = single_release,
};
-void kvmppc_create_vcpu_debugfs(struct kvm_vcpu *vcpu, unsigned int id)
+void kvmppc_create_vcpu_debugfs_e500(struct kvm_vcpu *vcpu,
+ struct dentry *debugfs_dentry)
{
- static char dbg_fname[50];
- struct dentry *debugfs_file;
-
- snprintf(dbg_fname, sizeof(dbg_fname), "vm%u_vcpu%u_timing",
- current->pid, id);
- debugfs_file = debugfs_create_file(dbg_fname, 0666, kvm_debugfs_dir,
- vcpu, &kvmppc_exit_timing_fops);
-
- vcpu->arch.debugfs_exit_timing = debugfs_file;
-}
-
-void kvmppc_remove_vcpu_debugfs(struct kvm_vcpu *vcpu)
-{
- debugfs_remove(vcpu->arch.debugfs_exit_timing);
- vcpu->arch.debugfs_exit_timing = NULL;
+ debugfs_create_file("timing", 0666, debugfs_dentry,
+ vcpu, &kvmppc_exit_timing_fops);
}
--
2.30.2
^ permalink raw reply related
* [PATCH] powerpc/kvm: no need to initialise statics to 0
From: Jason Wang @ 2021-12-20 3:02 UTC (permalink / raw)
To: mpe; +Cc: paulus, linuxppc-dev, wangborong, linux-kernel
Static variables do not need to be initialised to 0, because compiler
will initialise all uninitialised statics to 0. Thus, remove the
unneeded initialization.
Signed-off-by: Jason Wang <wangborong@cdjrlc.com>
---
arch/powerpc/kvm/book3s_64_mmu_host.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
index c3e31fef0be1..1ae09992c9ea 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
@@ -228,7 +228,7 @@ static struct kvmppc_sid_map *create_sid_map(struct kvm_vcpu *vcpu, u64 gvsid)
struct kvmppc_sid_map *map;
struct kvmppc_vcpu_book3s *vcpu_book3s = to_book3s(vcpu);
u16 sid_map_mask;
- static int backwards_map = 0;
+ static int backwards_map;
if (kvmppc_get_msr(vcpu) & MSR_PR)
gvsid |= VSID_PR;
--
2.34.1
^ permalink raw reply related
* [PATCH] powerpc: use strscpy to copy strings
From: Jason Wang @ 2021-12-20 3:24 UTC (permalink / raw)
To: mpe; +Cc: paulus, linuxppc-dev, wangborong, linux-kernel
The strlcpy should not be used because it doesn't limit the source
length. So that it will lead some potential bugs.
But the strscpy doesn't require reading memory from the src string
beyond the specified "count" bytes, and since the return value is
easier to error-check than strlcpy()'s. In addition, the implementation
is robust to the string changing out from underneath it, unlike the
current strlcpy() implementation.
Thus, replace strlcpy with strscpy.
Signed-off-by: Jason Wang <wangborong@cdjrlc.com>
---
arch/powerpc/platforms/pasemi/misc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/pasemi/misc.c b/arch/powerpc/platforms/pasemi/misc.c
index 1bf65d02d3ba..06a1ffd43bfe 100644
--- a/arch/powerpc/platforms/pasemi/misc.c
+++ b/arch/powerpc/platforms/pasemi/misc.c
@@ -35,7 +35,7 @@ static int __init find_i2c_driver(struct device_node *node,
for (i = 0; i < ARRAY_SIZE(i2c_devices); i++) {
if (!of_device_is_compatible(node, i2c_devices[i].of_device))
continue;
- if (strlcpy(info->type, i2c_devices[i].i2c_type,
+ if (strscpy(info->type, i2c_devices[i].i2c_type,
I2C_NAME_SIZE) >= I2C_NAME_SIZE)
return -ENOMEM;
return 0;
--
2.34.1
^ permalink raw reply related
* Re: [PATCH] tpm: Fix kexec crash due to access to ops NULL pointer (powerpc)
From: Sachin Sant @ 2021-12-20 5:17 UTC (permalink / raw)
To: Stefan Berger
Cc: Korrapati.Likhitha, pavrampu, linux-security-module, linux-kernel,
jgg, jarkko, gcwilson, linux-integrity, linuxppc-dev, peterhuewe
In-Reply-To: <20211212012804.1555661-1-stefanb@linux.ibm.com>
> On 12-Dec-2021, at 6:58 AM, Stefan Berger <stefanb@linux.ibm.com> wrote:
>
> Fix the following crash on kexec by checking chip->ops for a NULL pointer
> in tpm_chip_start() and returning an error code if this is the case.
>
> BUG: Kernel NULL pointer dereference on read at 0x00000060
> Faulting instruction address: 0xc00000000099a06c
> Oops: Kernel access of bad area, sig: 11 [#1]
> ...
> NIP [c00000000099a06c] tpm_chip_start+0x2c/0x140
> LR [c00000000099a808] tpm_chip_unregister+0x108/0x170
> Call Trace:
> [c0000000188bfa00] [c000000002b03930] fw_devlink_strict+0x0/0x8 (unreliable)
> [c0000000188bfa30] [c00000000099a808] tpm_chip_unregister+0x108/0x170
> [c0000000188bfa70] [c0000000009a3874] tpm_ibmvtpm_remove+0x34/0x130
> [c0000000188bfae0] [c000000000110dbc] vio_bus_remove+0x5c/0xb0
> [c0000000188bfb20] [c0000000009bc154] device_shutdown+0x1d4/0x3a8
> [c0000000188bfbc0] [c000000000196e14] kernel_restart_prepare+0x54/0x70
>
> The referenced patch below introduced a function to shut down the VIO bus.
> The bus shutdown now calls tpm_del_char_device (via tpm_chip_unregister)
> after a call to tpm_class_shutdown, which already set chip->ops to NULL.
> The crash occurrs when tpm_del_char_device calls tpm_chip_start with the
> chip->ops NULL pointer.
>
> Fixes: 39d0099f9439 ("powerpc/pseries: Add shutdown() to vio_driver and vio_bus")
> Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
> ---
With the patch applied, kexec with vTPM works as expected.
Tested-by: Sachin Sant <sachinp@linux.vnet.ibm.com>
Thanks
-Sachin
^ permalink raw reply
* Re: [RFC PATCH] powerpc/signal: sanitise PT_NIP and sa_handler low bits
From: Nicholas Piggin @ 2021-12-20 5:28 UTC (permalink / raw)
To: Sachin Sant; +Cc: linuxppc-dev
In-Reply-To: <BE11B400-C0DF-4478-9728-327C9232B75E@linux.vnet.ibm.com>
Excerpts from Sachin Sant's message of December 15, 2021 8:49 pm:
>
>> Reported-by: Sachin Sant <sachinp@linux.vnet.ibm.com>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
>> I'm not entirely sure about the 32-bit / compat part. Or the 64-bit for
>> that matter except that it does seem to fix the bug caused by the test
>> program.
>>
>> Thanks,
>> Nick
>>
>> arch/powerpc/kernel/signal_32.c | 23 ++++++++++++++++-------
>> arch/powerpc/kernel/signal_64.c | 17 ++++++++++++-----
>> 2 files changed, 28 insertions(+), 12 deletions(-)
>
> Sorry for the delayed feedback. I was observing confusing test results
> so had to be sure.
>
> Test results are against 5.16.0-rc5-03218-g798527287598 (powerpc/merge)
>
> I ran some extended set of kernel self tests (from powerpc/signal and
> powerpc/security) on POWER8, POWER9 and POWER10 configs.
>
> On POWER8 & POWER10 LPAR with this fix tests ran successfully.
>
> on POWER9 PowerNV with the fix and following set of configs
>
> CONFIG_PPC_IRQ_SOFT_MASK_DEBUG=y
> CONFIG_PPC_RFI_SRR_DEBUG=y
>
> the tests ran successfully.
>
> But with the fix and following set of configs
>
> CONFIG_PPC_IRQ_SOFT_MASK_DEBUG=y
> CONFIG_PPC_RFI_SRR_DEBUG=n
>
> I still a warning and then a crash:
>
> [ 550.568588] count-cache-flush: hardware flush enabled.
> [ 550.568593] link-stack-flush: software flush enabled.
> [ 550.569604] ------------[ cut here ]------------
> [ 550.569611] WARNING: CPU: 21 PID: 3784 at arch/powerpc/kernel/irq.c:288 arch_local_irq_restore+0x22c/0x230
> [ 550.569625] Modules linked in: nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables rfkill libcrc32c nfnetlink i2c_dev rpcrdma sunrpc ib_umad rdma_ucm ib_srpt ib_isert iscsi_target_mod target_core_mod ib_ipoib ib_iser rdma_cm iw_cm libiscsi ib_cm scsi_transport_iscsi mlx5_ib ib_uverbs dm_mod ib_core ses enclosure tpm_i2c_nuvoton i2c_opal ipmi_powernv xts ipmi_devintf uio_pdrv_genirq vmx_crypto ipmi_msghandler i2c_core opal_prd uio ibmpowernv leds_powernv powernv_op_panel sch_fq_codel ip_tables ext4 mbcache jbd2 mlx5_core sd_mod t10_pi sg mpt3sas ipr tg3 libata mlxfw psample raid_class ptp scsi_transport_sas pps_core fuse
> [ 550.569752] CPU: 21 PID: 3784 Comm: NetworkManager Kdump: loaded Not tainted 5.16.0-rc5-03218-g798527287598 #8
> [ 550.569765] NIP: c0000000000171dc LR: c000000000033240 CTR: c000000000cf1260
> [ 550.569774] REGS: c000000ae08bbab0 TRAP: 0700 Not tainted (5.16.0-rc5-03218-g798527287598)
> [ 550.569784] MSR: 9000000000021031 <SF,HV,ME,IR,DR,LE> CR: 28004444 XER: 20040000
> [ 550.569802] CFAR: c00000000001704c IRQMASK: 1
> [ 550.569802] GPR00: c000000000033240 c000000ae08bbd50 c000000002a10600 0000000000000000
> [ 550.569802] GPR04: c000000ae08bbe80 00007fffaea1ece0 0000000000000000 0000000000000000
> [ 550.569802] GPR08: 0000000000000002 0000000000000000 0000000000000002 0000000001f3fb0c
> [ 550.569802] GPR12: 0000000000004000 c000005fff7c9080 0000000000000000 0000000000000000
> [ 550.569802] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 550.569802] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 550.569802] GPR24: 0000000000000002 0000000000000001 0000000002002000 0000000002802000
> [ 550.569802] GPR28: 0000000000000000 0000000000000800 c000000ae08bbe80 0000000000040080
> [ 550.569899] NIP [c0000000000171dc] arch_local_irq_restore+0x22c/0x230
> [ 550.569909] LR [c000000000033240] interrupt_exit_user_prepare_main+0x150/0x260
> [ 550.569919] Call Trace:
> [ 550.569925] [c000000ae08bbd80] [c000000000033240] interrupt_exit_user_prepare_main+0x150/0x260
> [ 550.569937] [c000000ae08bbde0] [c000000000033744] syscall_exit_prepare+0x74/0x150
> [ 550.569948] [c000000ae08bbe10] [c00000000000c758] system_call_common+0xf8/0x268
Yeah this looks like a different issue. Is there a test running which
flips the security mitigations rapidly? There is a race window with
the the static branch causing exit_must_hard_disable() returning two
different values.
We should update they key while single threaded AFAIKS.
Thanks,
Nick
---
diff --git a/arch/powerpc/lib/feature-fixups.c b/arch/powerpc/lib/feature-fixups.c
index 57c6bb802f6c..a7cb317e7039 100644
--- a/arch/powerpc/lib/feature-fixups.c
+++ b/arch/powerpc/lib/feature-fixups.c
@@ -232,11 +232,22 @@ static DEFINE_MUTEX(exit_flush_lock);
static int __do_stf_barrier_fixups(void *data)
{
- enum stf_barrier_type *types = data;
+ enum stf_barrier_type types = *(enum stf_barrier_type *)data;
do_stf_entry_barrier_fixups(*types);
do_stf_exit_barrier_fixups(*types);
+ if ((types & STF_BARRIER_FALLBACK) || (types & STF_BARRIER_SYNC_ORI))
+ stf_exit_reentrant = false;
+ else
+ stf_exit_reentrant = true;
+
+ if (stf_exit_reentrant && rfi_exit_reentrant)
+ static_branch_disable(&interrupt_exit_not_reentrant);
+ else
+ static_branch_enable(&interrupt_exit_not_reentrant);
+
+
return 0;
}
@@ -257,18 +268,9 @@ void do_stf_barrier_fixups(enum stf_barrier_type types)
// Prevent static key update races with do_rfi_flush_fixups()
mutex_lock(&exit_flush_lock);
- static_branch_enable(&interrupt_exit_not_reentrant);
stop_machine(__do_stf_barrier_fixups, &types, NULL);
- if ((types & STF_BARRIER_FALLBACK) || (types & STF_BARRIER_SYNC_ORI))
- stf_exit_reentrant = false;
- else
- stf_exit_reentrant = true;
-
- if (stf_exit_reentrant && rfi_exit_reentrant)
- static_branch_disable(&interrupt_exit_not_reentrant);
-
mutex_unlock(&exit_flush_lock);
}
@@ -472,6 +474,16 @@ static int __do_rfi_flush_fixups(void *data)
patch_instruction(dest + 2, ppc_inst(instrs[2]));
}
+ if (types & L1D_FLUSH_FALLBACK)
+ rfi_exit_reentrant = false;
+ else
+ rfi_exit_reentrant = true;
+
+ if (stf_exit_reentrant && rfi_exit_reentrant)
+ static_branch_disable(&interrupt_exit_not_reentrant);
+ else
+ static_branch_enable(&interrupt_exit_not_reentrant);
+
printk(KERN_DEBUG "rfi-flush: patched %d locations (%s flush)\n", i,
(types == L1D_FLUSH_NONE) ? "no" :
(types == L1D_FLUSH_FALLBACK) ? "fallback displacement" :
@@ -495,18 +507,9 @@ void do_rfi_flush_fixups(enum l1d_flush_type types)
// Prevent static key update races with do_stf_barrier_fixups()
mutex_lock(&exit_flush_lock);
- static_branch_enable(&interrupt_exit_not_reentrant);
stop_machine(__do_rfi_flush_fixups, &types, NULL);
- if (types & L1D_FLUSH_FALLBACK)
- rfi_exit_reentrant = false;
- else
- rfi_exit_reentrant = true;
-
- if (stf_exit_reentrant && rfi_exit_reentrant)
- static_branch_disable(&interrupt_exit_not_reentrant);
-
mutex_unlock(&exit_flush_lock);
}
^ permalink raw reply related
* Re: [RFC PATCH] powerpc/signal: sanitise PT_NIP and sa_handler low bits
From: Sachin Sant @ 2021-12-20 7:11 UTC (permalink / raw)
To: Nicholas Piggin; +Cc: linuxppc-dev
In-Reply-To: <1639977910.h19vppmaz3.astroid@bobo.none>
>> [ 550.569802] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 550.569802] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 550.569802] GPR24: 0000000000000002 0000000000000001 0000000002002000 0000000002802000
>> [ 550.569802] GPR28: 0000000000000000 0000000000000800 c000000ae08bbe80 0000000000040080
>> [ 550.569899] NIP [c0000000000171dc] arch_local_irq_restore+0x22c/0x230
>> [ 550.569909] LR [c000000000033240] interrupt_exit_user_prepare_main+0x150/0x260
>> [ 550.569919] Call Trace:
>> [ 550.569925] [c000000ae08bbd80] [c000000000033240] interrupt_exit_user_prepare_main+0x150/0x260
>> [ 550.569937] [c000000ae08bbde0] [c000000000033744] syscall_exit_prepare+0x74/0x150
>> [ 550.569948] [c000000ae08bbe10] [c00000000000c758] system_call_common+0xf8/0x268
>
> Yeah this looks like a different issue. Is there a test running which
> flips the security mitigations rapidly? There is a race window with
Yes, powerpc/security/mitigation-patching.sh. This test enables/disables
various supported mitigations (parallel execution).
> the the static branch causing exit_must_hard_disable() returning two
> different values.
>
> We should update they key while single threaded AFAIKS.
Thanks. I tested with this fix. The test ran correctly without a crash.
> diff --git a/arch/powerpc/lib/feature-fixups.c b/arch/powerpc/lib/feature-fixups.c
> index 57c6bb802f6c..a7cb317e7039 100644
> --- a/arch/powerpc/lib/feature-fixups.c
> +++ b/arch/powerpc/lib/feature-fixups.c
> @@ -232,11 +232,22 @@ static DEFINE_MUTEX(exit_flush_lock);
>
> static int __do_stf_barrier_fixups(void *data)
> {
> - enum stf_barrier_type *types = data;
> + enum stf_barrier_type types = *(enum stf_barrier_type *)data;
>
> do_stf_entry_barrier_fixups(*types);
> do_stf_exit_barrier_fixups(*types);
>
*types should be changed to “types” to avoid build failure.
^ permalink raw reply
* Re: [PATCH kernel v4] KVM: PPC: Merge powerpc's debugfs entry content into generic entry
From: Cédric Le Goater @ 2021-12-20 7:29 UTC (permalink / raw)
To: Alexey Kardashevskiy, linuxppc-dev
Cc: kvm-ppc, kvm, David Stevens, Nicholas Piggin, Fabiano Rosas
In-Reply-To: <20211220012351.2719879-1-aik@ozlabs.ru>
On 12/20/21 02:23, Alexey Kardashevskiy wrote:
> At the moment KVM on PPC creates 4 types of entries under the kvm debugfs:
> 1) "%pid-%fd" per a KVM instance (for all platforms);
> 2) "vm%pid" (for PPC Book3s HV KVM);
> 3) "vm%u_vcpu%u_timing" (for PPC Book3e KVM);
> 4) "kvm-xive-%p" (for XIVE PPC Book3s KVM, the same for XICS);
>
> The problem with this is that multiple VMs per process is not allowed for
> 2) and 3) which makes it possible for userspace to trigger errors when
> creating duplicated debugfs entries.
>
> This merges all these into 1).
>
> This defines kvm_arch_create_kvm_debugfs() similar to
> kvm_arch_create_vcpu_debugfs().
>
> This defines 2 hooks in kvmppc_ops that allow specific KVM implementations
> add necessary entries, this adds the _e500 suffix to
> kvmppc_create_vcpu_debugfs_e500() to make it clear what platform it is for.
>
> This makes use of already existing kvm_arch_create_vcpu_debugfs() on PPC.
>
> This removes no more used debugfs_dir pointers from PPC kvm_arch structs.
>
> This stops removing vcpu entries as once created vcpus stay around
> for the entire life of a VM and removed when the KVM instance is closed,
> see commit d56f5136b010 ("KVM: let kvm_destroy_vm_debugfs clean up vCPU
> debugfs directories").
>
> Suggested-by: Fabiano Rosas <farosas@linux.ibm.com>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
One comment below.
> ---
> Changes:
> v4:
> * added "kvm-xive-%p"
>
> v3:
> * reworked commit log, especially, the bit about removing vcpus
>
> v2:
> * handled powerpc-booke
> * s/kvm/vm/ in arch hooks
> ---
> arch/powerpc/include/asm/kvm_host.h | 6 ++---
> arch/powerpc/include/asm/kvm_ppc.h | 2 ++
> arch/powerpc/kvm/timing.h | 9 ++++----
> arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +-
> arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +-
> arch/powerpc/kvm/book3s_hv.c | 31 ++++++++++----------------
> arch/powerpc/kvm/book3s_xics.c | 13 ++---------
> arch/powerpc/kvm/book3s_xive.c | 13 ++---------
> arch/powerpc/kvm/book3s_xive_native.c | 13 ++---------
> arch/powerpc/kvm/e500.c | 1 +
> arch/powerpc/kvm/e500mc.c | 1 +
> arch/powerpc/kvm/powerpc.c | 16 ++++++++++---
> arch/powerpc/kvm/timing.c | 20 ++++-------------
> 13 files changed, 47 insertions(+), 82 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
> index 17263276189e..f5e14fa683f4 100644
> --- a/arch/powerpc/include/asm/kvm_host.h
> +++ b/arch/powerpc/include/asm/kvm_host.h
> @@ -26,6 +26,8 @@
> #include <asm/hvcall.h>
> #include <asm/mce.h>
>
> +#define __KVM_HAVE_ARCH_VCPU_DEBUGFS
> +
> #define KVM_MAX_VCPUS NR_CPUS
> #define KVM_MAX_VCORES NR_CPUS
>
> @@ -295,7 +297,6 @@ struct kvm_arch {
> bool dawr1_enabled;
> pgd_t *pgtable;
> u64 process_table;
> - struct dentry *debugfs_dir;
> struct kvm_resize_hpt *resize_hpt; /* protected by kvm->lock */
> #endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */
> #ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
> @@ -673,7 +674,6 @@ struct kvm_vcpu_arch {
> u64 timing_min_duration[__NUMBER_OF_KVM_EXIT_TYPES];
> u64 timing_max_duration[__NUMBER_OF_KVM_EXIT_TYPES];
> u64 timing_last_exit;
> - struct dentry *debugfs_exit_timing;
> #endif
>
> #ifdef CONFIG_PPC_BOOK3S
> @@ -829,8 +829,6 @@ struct kvm_vcpu_arch {
> struct kvmhv_tb_accumulator rm_exit; /* real-mode exit code */
> struct kvmhv_tb_accumulator guest_time; /* guest execution */
> struct kvmhv_tb_accumulator cede_time; /* time napping inside guest */
> -
> - struct dentry *debugfs_dir;
> #endif /* CONFIG_KVM_BOOK3S_HV_EXIT_TIMING */
> };
>
> diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
> index 33db83b82fbd..d2b192dea0d2 100644
> --- a/arch/powerpc/include/asm/kvm_ppc.h
> +++ b/arch/powerpc/include/asm/kvm_ppc.h
> @@ -316,6 +316,8 @@ struct kvmppc_ops {
> int (*svm_off)(struct kvm *kvm);
> int (*enable_dawr1)(struct kvm *kvm);
> bool (*hash_v3_possible)(void);
> + int (*create_vm_debugfs)(struct kvm *kvm);
> + int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry);
> };
>
> extern struct kvmppc_ops *kvmppc_hv_ops;
> diff --git a/arch/powerpc/kvm/timing.h b/arch/powerpc/kvm/timing.h
> index feef7885ba82..493a7d510fd5 100644
> --- a/arch/powerpc/kvm/timing.h
> +++ b/arch/powerpc/kvm/timing.h
> @@ -14,8 +14,8 @@
> #ifdef CONFIG_KVM_EXIT_TIMING
> void kvmppc_init_timing_stats(struct kvm_vcpu *vcpu);
> void kvmppc_update_timing_stats(struct kvm_vcpu *vcpu);
> -void kvmppc_create_vcpu_debugfs(struct kvm_vcpu *vcpu, unsigned int id);
> -void kvmppc_remove_vcpu_debugfs(struct kvm_vcpu *vcpu);
> +void kvmppc_create_vcpu_debugfs_e500(struct kvm_vcpu *vcpu,
> + struct dentry *debugfs_dentry);
>
> static inline void kvmppc_set_exit_type(struct kvm_vcpu *vcpu, int type)
> {
> @@ -26,9 +26,8 @@ static inline void kvmppc_set_exit_type(struct kvm_vcpu *vcpu, int type)
> /* if exit timing is not configured there is no need to build the c file */
> static inline void kvmppc_init_timing_stats(struct kvm_vcpu *vcpu) {}
> static inline void kvmppc_update_timing_stats(struct kvm_vcpu *vcpu) {}
> -static inline void kvmppc_create_vcpu_debugfs(struct kvm_vcpu *vcpu,
> - unsigned int id) {}
> -static inline void kvmppc_remove_vcpu_debugfs(struct kvm_vcpu *vcpu) {}
> +static inline void kvmppc_create_vcpu_debugfs_e500(struct kvm_vcpu *vcpu,
> + struct dentry *debugfs_dentry) {}
> static inline void kvmppc_set_exit_type(struct kvm_vcpu *vcpu, int type) {}
> #endif /* CONFIG_KVM_EXIT_TIMING */
>
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index c63e263312a4..33dae253a0ac 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> @@ -2112,7 +2112,7 @@ static const struct file_operations debugfs_htab_fops = {
>
> void kvmppc_mmu_debugfs_init(struct kvm *kvm)
> {
> - debugfs_create_file("htab", 0400, kvm->arch.debugfs_dir, kvm,
> + debugfs_create_file("htab", 0400, kvm->debugfs_dentry, kvm,
> &debugfs_htab_fops);
> }
>
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c
> index 8cebe5542256..e4ce2a35483f 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c
> @@ -1454,7 +1454,7 @@ static const struct file_operations debugfs_radix_fops = {
>
> void kvmhv_radix_debugfs_init(struct kvm *kvm)
> {
> - debugfs_create_file("radix", 0400, kvm->arch.debugfs_dir, kvm,
> + debugfs_create_file("radix", 0400, kvm->debugfs_dentry, kvm,
> &debugfs_radix_fops);
> }
>
> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
> index bf1eb1160ae2..4c52541b6f37 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -2768,20 +2768,17 @@ static const struct file_operations debugfs_timings_ops = {
> };
>
> /* Create a debugfs directory for the vcpu */
> -static void debugfs_vcpu_init(struct kvm_vcpu *vcpu, unsigned int id)
> +static int kvmppc_arch_create_vcpu_debugfs_hv(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry)
> {
> - char buf[16];
> - struct kvm *kvm = vcpu->kvm;
> -
> - snprintf(buf, sizeof(buf), "vcpu%u", id);
> - vcpu->arch.debugfs_dir = debugfs_create_dir(buf, kvm->arch.debugfs_dir);
> - debugfs_create_file("timings", 0444, vcpu->arch.debugfs_dir, vcpu,
> + debugfs_create_file("timings", 0444, debugfs_dentry, vcpu,
> &debugfs_timings_ops);
> + return 0;
> }
>
> #else /* CONFIG_KVM_BOOK3S_HV_EXIT_TIMING */
> -static void debugfs_vcpu_init(struct kvm_vcpu *vcpu, unsigned int id)
> +static int kvmppc_arch_create_vcpu_debugfs_hv(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry)
> {
> + return 0;
> }
> #endif /* CONFIG_KVM_BOOK3S_HV_EXIT_TIMING */
>
> @@ -2904,8 +2901,6 @@ static int kvmppc_core_vcpu_create_hv(struct kvm_vcpu *vcpu)
> vcpu->arch.cpu_type = KVM_CPU_3S_64;
> kvmppc_sanity_check(vcpu);
>
> - debugfs_vcpu_init(vcpu, id);
> -
> return 0;
> }
>
> @@ -5226,7 +5221,6 @@ void kvmppc_free_host_rm_ops(void)
> static int kvmppc_core_init_vm_hv(struct kvm *kvm)
> {
> unsigned long lpcr, lpid;
> - char buf[32];
> int ret;
>
> mutex_init(&kvm->arch.uvmem_lock);
> @@ -5359,15 +5353,14 @@ static int kvmppc_core_init_vm_hv(struct kvm *kvm)
> kvm->arch.smt_mode = 1;
> kvm->arch.emul_smt_mode = 1;
>
> - /*
> - * Create a debugfs directory for the VM
> - */
> - snprintf(buf, sizeof(buf), "vm%d", current->pid);
> - kvm->arch.debugfs_dir = debugfs_create_dir(buf, kvm_debugfs_dir);
> + return 0;
> +}
> +
> +static int kvmppc_arch_create_vm_debugfs_hv(struct kvm *kvm)
> +{
> kvmppc_mmu_debugfs_init(kvm);
> if (radix_enabled())
> kvmhv_radix_debugfs_init(kvm);
> -
> return 0;
> }
>
> @@ -5382,8 +5375,6 @@ static void kvmppc_free_vcores(struct kvm *kvm)
>
> static void kvmppc_core_destroy_vm_hv(struct kvm *kvm)
> {
> - debugfs_remove_recursive(kvm->arch.debugfs_dir);
> -
> if (!cpu_has_feature(CPU_FTR_ARCH_300))
> kvm_hv_vm_deactivated();
>
> @@ -6044,6 +6035,8 @@ static struct kvmppc_ops kvm_ops_hv = {
> .svm_off = kvmhv_svm_off,
> .enable_dawr1 = kvmhv_enable_dawr1,
> .hash_v3_possible = kvmppc_hash_v3_possible,
> + .create_vcpu_debugfs = kvmppc_arch_create_vcpu_debugfs_hv,
> + .create_vm_debugfs = kvmppc_arch_create_vm_debugfs_hv,
> };
>
> static int kvm_init_subcore_bitmap(void)
> diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c
> index ebd5d920de8c..3dfa9285e4a4 100644
> --- a/arch/powerpc/kvm/book3s_xics.c
> +++ b/arch/powerpc/kvm/book3s_xics.c
> @@ -1016,19 +1016,10 @@ DEFINE_SHOW_ATTRIBUTE(xics_debug);
>
> static void xics_debugfs_init(struct kvmppc_xics *xics)
> {
> - char *name;
> -
> - name = kasprintf(GFP_KERNEL, "kvm-xics-%p", xics);
> - if (!name) {
> - pr_err("%s: no memory for name\n", __func__);
> - return;
> - }
> -
> - xics->dentry = debugfs_create_file(name, 0444, arch_debugfs_dir,
> + xics->dentry = debugfs_create_file("xics", 0444, xics->kvm->debugfs_dentry,
> xics, &xics_debug_fops);
>
> - pr_debug("%s: created %s\n", __func__, name);
> - kfree(name);
> + pr_debug("%s: created\n", __func__);
> }
>
> static struct kvmppc_ics *kvmppc_xics_create_ics(struct kvm *kvm,
> diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c
> index 225008882958..bb41afbb68fc 100644
> --- a/arch/powerpc/kvm/book3s_xive.c
> +++ b/arch/powerpc/kvm/book3s_xive.c
> @@ -2351,19 +2351,10 @@ DEFINE_SHOW_ATTRIBUTE(xive_debug);
>
> static void xive_debugfs_init(struct kvmppc_xive *xive)
> {
> - char *name;
> -
> - name = kasprintf(GFP_KERNEL, "kvm-xive-%p", xive);
> - if (!name) {
> - pr_err("%s: no memory for name\n", __func__);
> - return;
> - }
> -
> - xive->dentry = debugfs_create_file(name, S_IRUGO, arch_debugfs_dir,
> + xive->dentry = debugfs_create_file("xive", S_IRUGO, xive->kvm->debugfs_dentry,
> xive, &xive_debug_fops);
The KVM XIVE device implements a "xics-on-xive" interface, the XICS hcalls
on top of the XIVE native PowerNV (OPAL) interface, and ...
> - pr_debug("%s: created %s\n", __func__, name);
> - kfree(name);
> + pr_debug("%s: created\n", __func__);
> }
>
> static void kvmppc_xive_init(struct kvm_device *dev)
> diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c
> index 99db9ac49901..e86f5b6c2ae1 100644
> --- a/arch/powerpc/kvm/book3s_xive_native.c
> +++ b/arch/powerpc/kvm/book3s_xive_native.c
> @@ -1259,19 +1259,10 @@ DEFINE_SHOW_ATTRIBUTE(xive_native_debug);
>
> static void xive_native_debugfs_init(struct kvmppc_xive *xive)
> {
> - char *name;
> -
> - name = kasprintf(GFP_KERNEL, "kvm-xive-%p", xive);
> - if (!name) {
> - pr_err("%s: no memory for name\n", __func__);
> - return;
> - }
> -
> - xive->dentry = debugfs_create_file(name, 0444, arch_debugfs_dir,
> + xive->dentry = debugfs_create_file("xive", 0444, xive->kvm->debugfs_dentry,
> xive, &xive_native_debug_fops);
... the KVM XIVE *native* device implements a "xive" interface", the one
using MMIOs for interrupt management.
May be it's worth making the difference in the user interface ?
Thanks,
C.
>
> - pr_debug("%s: created %s\n", __func__, name);
> - kfree(name);
> + pr_debug("%s: created\n", __func__);
> }
>
> static void kvmppc_xive_native_init(struct kvm_device *dev)
> diff --git a/arch/powerpc/kvm/e500.c b/arch/powerpc/kvm/e500.c
> index 7e8b69015d20..c8b2b4478545 100644
> --- a/arch/powerpc/kvm/e500.c
> +++ b/arch/powerpc/kvm/e500.c
> @@ -495,6 +495,7 @@ static struct kvmppc_ops kvm_ops_e500 = {
> .emulate_op = kvmppc_core_emulate_op_e500,
> .emulate_mtspr = kvmppc_core_emulate_mtspr_e500,
> .emulate_mfspr = kvmppc_core_emulate_mfspr_e500,
> + .create_vcpu_debugfs = kvmppc_create_vcpu_debugfs_e500,
> };
>
> static int __init kvmppc_e500_init(void)
> diff --git a/arch/powerpc/kvm/e500mc.c b/arch/powerpc/kvm/e500mc.c
> index 1c189b5aadcc..fa0d8dbbe484 100644
> --- a/arch/powerpc/kvm/e500mc.c
> +++ b/arch/powerpc/kvm/e500mc.c
> @@ -381,6 +381,7 @@ static struct kvmppc_ops kvm_ops_e500mc = {
> .emulate_op = kvmppc_core_emulate_op_e500,
> .emulate_mtspr = kvmppc_core_emulate_mtspr_e500,
> .emulate_mfspr = kvmppc_core_emulate_mfspr_e500,
> + .create_vcpu_debugfs = kvmppc_create_vcpu_debugfs_e500,
> };
>
> static int __init kvmppc_e500mc_init(void)
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index a72920f4f221..2ea73dfcebb2 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -763,7 +763,6 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
> goto out_vcpu_uninit;
>
> vcpu->arch.waitp = &vcpu->wait;
> - kvmppc_create_vcpu_debugfs(vcpu, vcpu->vcpu_id);
> return 0;
>
> out_vcpu_uninit:
> @@ -780,8 +779,6 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
> /* Make sure we're not using the vcpu anymore */
> hrtimer_cancel(&vcpu->arch.dec_timer);
>
> - kvmppc_remove_vcpu_debugfs(vcpu);
> -
> switch (vcpu->arch.irq_type) {
> case KVMPPC_IRQ_MPIC:
> kvmppc_mpic_disconnect_vcpu(vcpu->arch.mpic, vcpu);
> @@ -2505,3 +2502,16 @@ int kvm_arch_init(void *opaque)
> }
>
> EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_ppc_instr);
> +
> +void kvm_arch_create_vcpu_debugfs(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry)
> +{
> + if (vcpu->kvm->arch.kvm_ops->create_vcpu_debugfs)
> + vcpu->kvm->arch.kvm_ops->create_vcpu_debugfs(vcpu, debugfs_dentry);
> +}
> +
> +int kvm_arch_create_vm_debugfs(struct kvm *kvm)
> +{
> + if (kvm->arch.kvm_ops->create_vm_debugfs)
> + kvm->arch.kvm_ops->create_vm_debugfs(kvm);
> + return 0;
> +}
> diff --git a/arch/powerpc/kvm/timing.c b/arch/powerpc/kvm/timing.c
> index ba56a5cbba97..f6d472874c85 100644
> --- a/arch/powerpc/kvm/timing.c
> +++ b/arch/powerpc/kvm/timing.c
> @@ -204,21 +204,9 @@ static const struct file_operations kvmppc_exit_timing_fops = {
> .release = single_release,
> };
>
> -void kvmppc_create_vcpu_debugfs(struct kvm_vcpu *vcpu, unsigned int id)
> +void kvmppc_create_vcpu_debugfs_e500(struct kvm_vcpu *vcpu,
> + struct dentry *debugfs_dentry)
> {
> - static char dbg_fname[50];
> - struct dentry *debugfs_file;
> -
> - snprintf(dbg_fname, sizeof(dbg_fname), "vm%u_vcpu%u_timing",
> - current->pid, id);
> - debugfs_file = debugfs_create_file(dbg_fname, 0666, kvm_debugfs_dir,
> - vcpu, &kvmppc_exit_timing_fops);
> -
> - vcpu->arch.debugfs_exit_timing = debugfs_file;
> -}
> -
> -void kvmppc_remove_vcpu_debugfs(struct kvm_vcpu *vcpu)
> -{
> - debugfs_remove(vcpu->arch.debugfs_exit_timing);
> - vcpu->arch.debugfs_exit_timing = NULL;
> + debugfs_create_file("timing", 0666, debugfs_dentry,
> + vcpu, &kvmppc_exit_timing_fops);
> }
>
^ permalink raw reply
* [PATCH 2/5] crypto: mips/sha - remove duplicate hash init function
From: Tianjia Zhang @ 2021-12-20 9:23 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, Thomas Bogendoerfer,
Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Alexander Gordeev, linux-crypto, linux-mips, linux-kernel,
linuxppc-dev, linux-s390, sparclinux
Cc: Tianjia Zhang
In-Reply-To: <20211220092318.5793-1-tianjia.zhang@linux.alibaba.com>
sha*_base_init() series functions has implemented the initialization
of the hash context, this commit use sha*_base_init() function to
replace repeated implementations.
Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
---
arch/mips/cavium-octeon/crypto/octeon-sha1.c | 17 +-------
.../mips/cavium-octeon/crypto/octeon-sha256.c | 39 ++-----------------
.../mips/cavium-octeon/crypto/octeon-sha512.c | 39 ++-----------------
3 files changed, 8 insertions(+), 87 deletions(-)
diff --git a/arch/mips/cavium-octeon/crypto/octeon-sha1.c b/arch/mips/cavium-octeon/crypto/octeon-sha1.c
index 30f1d75208a5..37a07b3c4568 100644
--- a/arch/mips/cavium-octeon/crypto/octeon-sha1.c
+++ b/arch/mips/cavium-octeon/crypto/octeon-sha1.c
@@ -15,6 +15,7 @@
#include <linux/mm.h>
#include <crypto/sha1.h>
+#include <crypto/sha1_base.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/module.h>
@@ -71,20 +72,6 @@ static void octeon_sha1_transform(const void *_block)
octeon_sha1_start(block[7]);
}
-static int octeon_sha1_init(struct shash_desc *desc)
-{
- struct sha1_state *sctx = shash_desc_ctx(desc);
-
- sctx->state[0] = SHA1_H0;
- sctx->state[1] = SHA1_H1;
- sctx->state[2] = SHA1_H2;
- sctx->state[3] = SHA1_H3;
- sctx->state[4] = SHA1_H4;
- sctx->count = 0;
-
- return 0;
-}
-
static void __octeon_sha1_update(struct sha1_state *sctx, const u8 *data,
unsigned int len)
{
@@ -200,7 +187,7 @@ static int octeon_sha1_import(struct shash_desc *desc, const void *in)
static struct shash_alg octeon_sha1_alg = {
.digestsize = SHA1_DIGEST_SIZE,
- .init = octeon_sha1_init,
+ .init = sha1_base_init,
.update = octeon_sha1_update,
.final = octeon_sha1_final,
.export = octeon_sha1_export,
diff --git a/arch/mips/cavium-octeon/crypto/octeon-sha256.c b/arch/mips/cavium-octeon/crypto/octeon-sha256.c
index 36cb92895d72..435e4a6e7f13 100644
--- a/arch/mips/cavium-octeon/crypto/octeon-sha256.c
+++ b/arch/mips/cavium-octeon/crypto/octeon-sha256.c
@@ -16,6 +16,7 @@
#include <linux/mm.h>
#include <crypto/sha2.h>
+#include <crypto/sha256_base.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/module.h>
@@ -63,40 +64,6 @@ static void octeon_sha256_transform(const void *_block)
octeon_sha256_start(block[7]);
}
-static int octeon_sha224_init(struct shash_desc *desc)
-{
- struct sha256_state *sctx = shash_desc_ctx(desc);
-
- sctx->state[0] = SHA224_H0;
- sctx->state[1] = SHA224_H1;
- sctx->state[2] = SHA224_H2;
- sctx->state[3] = SHA224_H3;
- sctx->state[4] = SHA224_H4;
- sctx->state[5] = SHA224_H5;
- sctx->state[6] = SHA224_H6;
- sctx->state[7] = SHA224_H7;
- sctx->count = 0;
-
- return 0;
-}
-
-static int octeon_sha256_init(struct shash_desc *desc)
-{
- struct sha256_state *sctx = shash_desc_ctx(desc);
-
- sctx->state[0] = SHA256_H0;
- sctx->state[1] = SHA256_H1;
- sctx->state[2] = SHA256_H2;
- sctx->state[3] = SHA256_H3;
- sctx->state[4] = SHA256_H4;
- sctx->state[5] = SHA256_H5;
- sctx->state[6] = SHA256_H6;
- sctx->state[7] = SHA256_H7;
- sctx->count = 0;
-
- return 0;
-}
-
static void __octeon_sha256_update(struct sha256_state *sctx, const u8 *data,
unsigned int len)
{
@@ -224,7 +191,7 @@ static int octeon_sha256_import(struct shash_desc *desc, const void *in)
static struct shash_alg octeon_sha256_algs[2] = { {
.digestsize = SHA256_DIGEST_SIZE,
- .init = octeon_sha256_init,
+ .init = sha256_base_init,
.update = octeon_sha256_update,
.final = octeon_sha256_final,
.export = octeon_sha256_export,
@@ -240,7 +207,7 @@ static struct shash_alg octeon_sha256_algs[2] = { {
}
}, {
.digestsize = SHA224_DIGEST_SIZE,
- .init = octeon_sha224_init,
+ .init = sha224_base_init,
.update = octeon_sha256_update,
.final = octeon_sha224_final,
.descsize = sizeof(struct sha256_state),
diff --git a/arch/mips/cavium-octeon/crypto/octeon-sha512.c b/arch/mips/cavium-octeon/crypto/octeon-sha512.c
index 359f039820d8..2dee9354e33f 100644
--- a/arch/mips/cavium-octeon/crypto/octeon-sha512.c
+++ b/arch/mips/cavium-octeon/crypto/octeon-sha512.c
@@ -15,6 +15,7 @@
#include <linux/mm.h>
#include <crypto/sha2.h>
+#include <crypto/sha512_base.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/module.h>
@@ -74,40 +75,6 @@ static void octeon_sha512_transform(const void *_block)
octeon_sha512_start(block[15]);
}
-static int octeon_sha512_init(struct shash_desc *desc)
-{
- struct sha512_state *sctx = shash_desc_ctx(desc);
-
- sctx->state[0] = SHA512_H0;
- sctx->state[1] = SHA512_H1;
- sctx->state[2] = SHA512_H2;
- sctx->state[3] = SHA512_H3;
- sctx->state[4] = SHA512_H4;
- sctx->state[5] = SHA512_H5;
- sctx->state[6] = SHA512_H6;
- sctx->state[7] = SHA512_H7;
- sctx->count[0] = sctx->count[1] = 0;
-
- return 0;
-}
-
-static int octeon_sha384_init(struct shash_desc *desc)
-{
- struct sha512_state *sctx = shash_desc_ctx(desc);
-
- sctx->state[0] = SHA384_H0;
- sctx->state[1] = SHA384_H1;
- sctx->state[2] = SHA384_H2;
- sctx->state[3] = SHA384_H3;
- sctx->state[4] = SHA384_H4;
- sctx->state[5] = SHA384_H5;
- sctx->state[6] = SHA384_H6;
- sctx->state[7] = SHA384_H7;
- sctx->count[0] = sctx->count[1] = 0;
-
- return 0;
-}
-
static void __octeon_sha512_update(struct sha512_state *sctx, const u8 *data,
unsigned int len)
{
@@ -223,7 +190,7 @@ static int octeon_sha384_final(struct shash_desc *desc, u8 *hash)
static struct shash_alg octeon_sha512_algs[2] = { {
.digestsize = SHA512_DIGEST_SIZE,
- .init = octeon_sha512_init,
+ .init = sha512_base_init,
.update = octeon_sha512_update,
.final = octeon_sha512_final,
.descsize = sizeof(struct sha512_state),
@@ -236,7 +203,7 @@ static struct shash_alg octeon_sha512_algs[2] = { {
}
}, {
.digestsize = SHA384_DIGEST_SIZE,
- .init = octeon_sha384_init,
+ .init = sha384_base_init,
.update = octeon_sha512_update,
.final = octeon_sha384_final,
.descsize = sizeof(struct sha512_state),
--
2.32.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox