* Re: [PATCH net v4 3/4] iavf: send MAC change request synchronously
From: Simon Horman @ 2026-04-27 16:43 UTC (permalink / raw)
To: jtornosm
Cc: 'Simon Horman', netdev, intel-wired-lan,
przemyslaw.kitszel, aleksandr.loktionov, jacob.e.keller,
jesse.brandeburg, anthony.l.nguyen, davem, edumazet, kuba, pabeni
In-Reply-To: <20260423130405.139568-4-jtornosm@redhat.com>
From: 'Simon Horman' <horms@kernel.org>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
iavf: send MAC change request synchronously
This commit updates the MAC address setting in the iavf driver to be
synchronous. It sends the virtchnl message directly and polls the admin
queue for the response, rather than relying on the watchdog task, to avoid
deadlocks when holding the netdev instance lock.
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
> index 67aa14350b1b01..bc5994bf2cd99d 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_main.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
[ ... ]
> @@ -1067,25 +1109,20 @@ static int iavf_set_mac(struct net_device *netdev, void *p)
> return -EADDRNOTAVAIL;
>
> ret = iavf_replace_primary_mac(adapter, addr->sa_data);
> -
> if (ret)
> return ret;
>
> - ret = wait_event_interruptible_timeout(adapter->vc_waitqueue,
> - iavf_is_mac_set_handled(netdev, addr->sa_data),
> - msecs_to_jiffies(2500));
> -
> - /* If ret < 0 then it means wait was interrupted.
> - * If ret == 0 then it means we got a timeout.
> - * else it means we got response for set MAC from PF,
> - * check if netdev MAC was updated to requested MAC,
> - * if yes then set MAC succeeded otherwise it failed return -EACCES
> - */
> - if (ret < 0)
> + ret = iavf_set_mac_sync(adapter, addr->sa_data);
> + if (ret) {
> + /* Rollback for local failures (timeout, send error, -EBUSY).
> + * Note: If PF rejects the request (sends error response),
> + * iavf_virtchnl_completion() automatically calls
> + * iavf_mac_add_reject(), ret=0, and this is not executed.
> + * Only local failures (no PF response received) need manual rollback.
> + */
> + iavf_mac_add_reject(adapter);
> return ret;
> -
> - if (!ret)
> - return -EAGAIN;
> + }
Does the manual rollback in iavf_set_mac() need to restore adapter->hw.mac.addr?
Before calling iavf_set_mac_sync(), iavf_replace_primary_mac() optimistically
updates adapter->hw.mac.addr to the new MAC address and unsets the is_primary
flag on the old MAC filter. If iavf_set_mac_sync() fails locally,
iavf_mac_add_reject() deletes the pending new MAC filter but doesn't restore
adapter->hw.mac.addr back to netdev->dev_addr.
By contrast, the PF-rejection path in iavf_virtchnl_completion() explicitly
restores it:
iavf_mac_add_reject(adapter);
/* restore administratively set MAC address */
ether_addr_copy(adapter->hw.mac.addr, netdev->dev_addr);
If the local failure was a timeout and a delayed success response arrives from
the PF later, could iavf_virtchnl_completion() blindly copy the corrupted
adapter->hw.mac.addr into netdev->dev_addr? This seems like it would silently
change the interface MAC to an address already deleted from the internal list.
Additionally, on subsequent MAC change attempts, wouldn't
iavf_replace_primary_mac() search for the old filter using the corrupted
adapter->hw.mac.addr, fail to find it, and permanently leak the old filter in
the PF's hardware tables?
[ ... ]
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
> index a52c100dcbc56d..d1afb8261c2412 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
[ ... ]
> @@ -2956,3 +2966,76 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
> } /* switch v_opcode */
> adapter->current_op = VIRTCHNL_OP_UNKNOWN;
> }
> +
> +/**
> + * iavf_poll_virtchnl_response - Poll admin queue for virtchnl response
> + * @adapter: adapter structure
> + * @condition: callback to check if desired response received
> + * @cond_data: context data passed to condition callback
> + * @timeout_ms: maximum time to wait in milliseconds
> + *
> + * Polls the admin queue and processes all incoming virtchnl messages.
> + * After processing each valid message, calls the condition callback to check
> + * if the expected response has been received. The callback receives the opcode
> + * of the processed message to identify which response was received. Continues
> + * polling until the callback returns true or timeout expires.
> + * Clear current_op on timeout to prevent permanent -EBUSY state.
> + * Caller must hold netdev_lock. This can sleep for up to timeout_ms while
> + * polling hardware.
> + *
> + * Return: 0 on success (condition met), -EAGAIN on timeout, or error code
> + **/
> +int iavf_poll_virtchnl_response(struct iavf_adapter *adapter,
> + bool (*condition)(struct iavf_adapter *adapter,
> + const void *data,
> + enum virtchnl_ops v_op),
> + const void *cond_data,
> + unsigned int timeout_ms)
> +{
[ ... ]
> + if (iavf_clean_arq_element(hw, &event, &pending) == IAVF_SUCCESS) {
> + received_op = (enum virtchnl_ops)le32_to_cpu(event.desc.cookie_high);
> + if (received_op != VIRTCHNL_OP_UNKNOWN) {
> + v_retval = le32_to_cpu(event.desc.cookie_low);
> +
> + iavf_virtchnl_completion(adapter, received_op,
> + (enum iavf_status)v_retval,
> + event.msg_buf, event.msg_len);
> +
> + if (condition(adapter, cond_data, received_op)) {
> + ret = 0;
> + break;
> + }
> + }
> +
> + memset(event.msg_buf, 0, IAVF_MAX_AQ_BUF_SIZE);
> +
> + if (pending)
> + continue;
> + }
> +
> + usleep_range(50, 75);
> + } while (time_before(jiffies, timeout));
> +
> + if (ret == -EAGAIN && adapter->current_op != VIRTCHNL_OP_UNKNOWN)
> + adapter->current_op = VIRTCHNL_OP_UNKNOWN;
Could clearing adapter->current_op on timeout cause overlapping virtchnl
commands and state corruption?
If the timeout expires and adapter->current_op is forced to
VIRTCHNL_OP_UNKNOWN, the VF watchdog might consider the admin queue free
and send a new command while the PF is still actively processing the
timed-out request.
If the PF eventually completes the timed-out request and sends a late response,
iavf_virtchnl_completion() will process it. At the end of
iavf_virtchnl_completion(), the driver unconditionally clears the state:
adapter->current_op = VIRTCHNL_OP_UNKNOWN;
Would this prematurely clear the tracking state for the newly in-flight
overlapping command, compounding the state machine corruption and allowing
even more commands to be sent concurrently?
> +
> + kfree(event.msg_buf);
> + return ret;
> +}
^ permalink raw reply
* Re: [PATCH v2 iproute2-next] utils: add fflush_monitor() helper
From: Eric Dumazet @ 2026-04-27 16:47 UTC (permalink / raw)
To: Stephen Hemminger
Cc: David Ahern, David S . Miller, Jakub Kicinski, Paolo Abeni,
netdev, eric.dumazet
In-Reply-To: <20260427093149.2cb7a215@phoenix.local>
On Mon, Apr 27, 2026 at 9:31 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Mon, 27 Apr 2026 08:19:53 +0000
> Eric Dumazet <edumazet@google.com> wrote:
>
> > Some fflush() calls only make sense for monitor programs.
> >
> > For other cases, forcing a flush is expensive.
> >
> > After this patch, ip, tc and ss are correctly buffering most of their
> > output when redirected to a file.
> >
> > Signed-off-by: Eric Dumazet <edumazet@google.com>
>
> What about pulling all the fflush from the leaf calls, and move the fflush
> to one common place. There are some of these that need to come back
> like the addr flush loop, this is just a semi-automated first pass.
SGTM, thanks.
^ permalink raw reply
* [PATCH v4 net-next 0/3] Implement PTP support in netdevsim
From: Maciek Machnikowski @ 2026-04-27 16:47 UTC (permalink / raw)
To: netdev
Cc: kuba, maciek, richardcochran, milena.olech, willemdebruijn.kernel,
andrew, vadim.fedorenko, horms
This patchset adds support to the PTP HW timestamping emulation in the
netdevsim. It uses existing binding between netdevsim and ptp_mock
driver to generate transmit and receive timestamps.
It also adds the selftest to verify the hw timestamping functionality
running over netdevsim.
v2:
- Added selftest/net/ptp.sh
- Modified ptp_mock to use spin_lock_bh
- Populate ethtool defaults using ethtool_op_get_ts_info
v3:
- Fixed shellcheck issues in the selftest/net/ptp.sh
- Added selftest/net/ptp.sh to the selftest/net/Makefile
- Modified ptp_mock to use spin_lock_irqsave
v4:
- Check if Rx timestamps are enabled before generating a timestamp
- Replace bash selftest script with a python one
- Optimized Tx timestamp generation
Maciek Machnikowski (3):
ptp_mock: Expose ptp_clock_info to external drivers
netdevsim: Implement basic ptp support
selftests:net: Implement ptp4l sync test using netdevsim
drivers/net/netdevsim/ethtool.c | 11 ++
drivers/net/netdevsim/netdev.c | 91 ++++++++++++
drivers/net/netdevsim/netdevsim.h | 1 +
drivers/ptp/ptp_mock.c | 26 ++--
include/linux/ptp_mock.h | 5 +
tools/testing/selftests/net/Makefile | 1 +
tools/testing/selftests/net/ptp.py | 184 +++++++++++++++++++++++++++
7 files changed, 311 insertions(+), 8 deletions(-)
create mode 100755 tools/testing/selftests/net/ptp.py
--
2.53.0
^ permalink raw reply
* [PATCH v4 net-next 1/3] ptp_mock: Expose ptp_clock_info to external drivers
From: Maciek Machnikowski @ 2026-04-27 16:47 UTC (permalink / raw)
To: netdev
Cc: kuba, maciek, richardcochran, milena.olech, willemdebruijn.kernel,
andrew, vadim.fedorenko, horms
In-Reply-To: <20260427164727.15418-1-maciek@machnikowski.net>
Allow exposing the ptp_clock_info of the ptp_mock to the external drivers.
Convert spinlocks to SLIS to allow gettime to be called from the netdevsim.
This is a prerequisite for implementing ptp support on netdevsim.
Co-developed-by: Milena Olech <milena.olech@intel.com>
Signed-off-by: Milena Olech <milena.olech@intel.com>
Signed-off-by: Maciek Machnikowski <maciek@machnikowski.net>
---
drivers/ptp/ptp_mock.c | 26 ++++++++++++++++++--------
include/linux/ptp_mock.h | 5 +++++
2 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/drivers/ptp/ptp_mock.c b/drivers/ptp/ptp_mock.c
index 4d66b6147121..7a4e5f3274a6 100644
--- a/drivers/ptp/ptp_mock.c
+++ b/drivers/ptp/ptp_mock.c
@@ -49,15 +49,16 @@ static u64 mock_phc_cc_read(struct cyclecounter *cc)
static int mock_phc_adjfine(struct ptp_clock_info *info, long scaled_ppm)
{
struct mock_phc *phc = info_to_phc(info);
+ unsigned long flags;
s64 adj;
adj = (s64)scaled_ppm << MOCK_PHC_FADJ_SHIFT;
adj = div_s64(adj, MOCK_PHC_FADJ_DENOMINATOR);
- spin_lock(&phc->lock);
+ spin_lock_irqsave(&phc->lock, flags);
timecounter_read(&phc->tc);
phc->cc.mult = MOCK_PHC_CC_MULT + adj;
- spin_unlock(&phc->lock);
+ spin_unlock_irqrestore(&phc->lock, flags);
return 0;
}
@@ -65,10 +66,11 @@ static int mock_phc_adjfine(struct ptp_clock_info *info, long scaled_ppm)
static int mock_phc_adjtime(struct ptp_clock_info *info, s64 delta)
{
struct mock_phc *phc = info_to_phc(info);
+ unsigned long flags;
- spin_lock(&phc->lock);
+ spin_lock_irqsave(&phc->lock, flags);
timecounter_adjtime(&phc->tc, delta);
- spin_unlock(&phc->lock);
+ spin_unlock_irqrestore(&phc->lock, flags);
return 0;
}
@@ -78,10 +80,11 @@ static int mock_phc_settime64(struct ptp_clock_info *info,
{
struct mock_phc *phc = info_to_phc(info);
u64 ns = timespec64_to_ns(ts);
+ unsigned long flags;
- spin_lock(&phc->lock);
+ spin_lock_irqsave(&phc->lock, flags);
timecounter_init(&phc->tc, &phc->cc, ns);
- spin_unlock(&phc->lock);
+ spin_unlock_irqrestore(&phc->lock, flags);
return 0;
}
@@ -89,11 +92,12 @@ static int mock_phc_settime64(struct ptp_clock_info *info,
static int mock_phc_gettime64(struct ptp_clock_info *info, struct timespec64 *ts)
{
struct mock_phc *phc = info_to_phc(info);
+ unsigned long flags;
u64 ns;
- spin_lock(&phc->lock);
+ spin_lock_irqsave(&phc->lock, flags);
ns = timecounter_read(&phc->tc);
- spin_unlock(&phc->lock);
+ spin_unlock_irqrestore(&phc->lock, flags);
*ts = ns_to_timespec64(ns);
@@ -171,5 +175,11 @@ void mock_phc_destroy(struct mock_phc *phc)
}
EXPORT_SYMBOL_GPL(mock_phc_destroy);
+struct ptp_clock_info *mock_phc_get_ptp_info(struct mock_phc *phc)
+{
+ return &phc->info;
+}
+EXPORT_SYMBOL_GPL(mock_phc_get_ptp_info);
+
MODULE_DESCRIPTION("Mock-up PTP Hardware Clock driver");
MODULE_LICENSE("GPL");
diff --git a/include/linux/ptp_mock.h b/include/linux/ptp_mock.h
index 72eb401034d9..e33188dec2b7 100644
--- a/include/linux/ptp_mock.h
+++ b/include/linux/ptp_mock.h
@@ -16,6 +16,7 @@ struct mock_phc;
struct mock_phc *mock_phc_create(struct device *dev);
void mock_phc_destroy(struct mock_phc *phc);
int mock_phc_index(struct mock_phc *phc);
+struct ptp_clock_info *mock_phc_get_ptp_info(struct mock_phc *phc);
#else
@@ -33,6 +34,10 @@ static inline int mock_phc_index(struct mock_phc *phc)
return -1;
}
+static inline struct ptp_clock_info *mock_phc_get_ptp_info(struct mock_phc *phc)
+{
+ return NULL;
+}
#endif
#endif /* _PTP_MOCK_H_ */
--
2.53.0
^ permalink raw reply related
* [PATCH v4 net-next 2/3] netdevsim: Implement basic ptp support
From: Maciek Machnikowski @ 2026-04-27 16:47 UTC (permalink / raw)
To: netdev
Cc: kuba, maciek, richardcochran, milena.olech, willemdebruijn.kernel,
andrew, vadim.fedorenko, horms
In-Reply-To: <20260427164727.15418-1-maciek@machnikowski.net>
Add support for virtual timestamping inside the netdevsim driver.
The implementation uses two attached ptp_mock clocks, reads the timestamps
of the ones attached either to the netdevsim or its peer and returns
timestamps using standard timestamps APIs.
This implementation enables running ptp4l on netdevsim adapters and
introduces a new ptp selftest.
Co-developed-by: Milena Olech <milena.olech@intel.com>
Signed-off-by: Milena Olech <milena.olech@intel.com>
Signed-off-by: Maciek Machnikowski <maciek@machnikowski.net>
---
drivers/net/netdevsim/ethtool.c | 11 ++++
drivers/net/netdevsim/netdev.c | 91 +++++++++++++++++++++++++++++++
drivers/net/netdevsim/netdevsim.h | 1 +
3 files changed, 103 insertions(+)
diff --git a/drivers/net/netdevsim/ethtool.c b/drivers/net/netdevsim/ethtool.c
index 36a201533aae..5b709033cc5f 100644
--- a/drivers/net/netdevsim/ethtool.c
+++ b/drivers/net/netdevsim/ethtool.c
@@ -200,7 +200,18 @@ static int nsim_get_ts_info(struct net_device *dev,
{
struct netdevsim *ns = netdev_priv(dev);
+ ethtool_op_get_ts_info(dev, info);
+
info->phc_index = mock_phc_index(ns->phc);
+ if (info->phc_index < 0)
+ return 0;
+
+ info->so_timestamping |= SOF_TIMESTAMPING_TX_HARDWARE |
+ SOF_TIMESTAMPING_RX_HARDWARE |
+ SOF_TIMESTAMPING_RAW_HARDWARE;
+
+ info->tx_types = BIT(HWTSTAMP_TX_OFF) | BIT(HWTSTAMP_TX_ON);
+ info->rx_filters = BIT(HWTSTAMP_FILTER_NONE) | BIT(HWTSTAMP_FILTER_ALL);
return 0;
}
diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c
index c71b8d116f18..af1783c88753 100644
--- a/drivers/net/netdevsim/netdev.c
+++ b/drivers/net/netdevsim/netdev.c
@@ -30,6 +30,8 @@
#include <net/rtnetlink.h>
#include <net/udp_tunnel.h>
#include <net/busy_poll.h>
+#include <linux/ptp_clock_kernel.h>
+#include <linux/timecounter.h>
#include "netdevsim.h"
@@ -122,7 +124,11 @@ static int nsim_forward_skb(struct net_device *tx_dev,
static netdev_tx_t nsim_start_xmit(struct sk_buff *skb, struct net_device *dev)
{
+ struct skb_shared_hwtstamps shhwtstamps = {};
struct netdevsim *ns = netdev_priv(dev);
+ struct ptp_clock_info *ptp_info;
+ struct timespec64 tx_ts, rx_ts;
+ struct sk_buff *skb_orig = skb;
struct skb_ext *psp_ext = NULL;
struct net_device *peer_dev;
unsigned int len = skb->len;
@@ -164,6 +170,36 @@ static netdev_tx_t nsim_start_xmit(struct sk_buff *skb, struct net_device *dev)
skb_linearize(skb);
skb_tx_timestamp(skb);
+
+ /* Generate RX timestamp using the peer's PHC if RX timestamping is enabled */
+ if (peer_ns->tstamp_config.rx_filter != HWTSTAMP_FILTER_NONE) {
+ ptp_info = mock_phc_get_ptp_info(peer_ns->phc);
+ ptp_info->gettime64(ptp_info, &rx_ts);
+ }
+
+ /* If TX hardware timestamping is enabled, generate and attach a TX timestamp */
+ if (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP &&
+ peer_ns->tstamp_config.tx_type == HWTSTAMP_TX_ON) {
+ ptp_info = mock_phc_get_ptp_info(ns->phc);
+ ptp_info->gettime64(ptp_info, &tx_ts);
+
+ /* Create a copy of the SKB to forward to peer and prevent
+ * from reporting incorrect TX timestamp when skb_hwtstamps is set.
+ */
+ skb = skb_copy(skb_orig, GFP_ATOMIC);
+ if (skb) {
+ shhwtstamps.hwtstamp = timespec64_to_ktime(tx_ts);
+ skb_tstamp_tx(skb_orig, &shhwtstamps);
+ consume_skb(skb_orig);
+ } else {
+ skb = skb_orig;
+ }
+ }
+
+ /* set the rx timestamp to the skb */
+ if (peer_ns->tstamp_config.rx_filter != HWTSTAMP_FILTER_NONE)
+ skb_hwtstamps(skb)->hwtstamp = timespec64_to_ktime(rx_ts);
+
if (unlikely(nsim_forward_skb(dev, peer_dev,
skb, rq, psp_ext) == NET_RX_DROP))
goto out_drop_cnt;
@@ -185,6 +221,59 @@ static netdev_tx_t nsim_start_xmit(struct sk_buff *skb, struct net_device *dev)
return NETDEV_TX_OK;
}
+static int nsim_set_ts_config(struct net_device *netdev,
+ struct kernel_hwtstamp_config *config,
+ struct netlink_ext_ack *extack)
+{
+ struct netdevsim *ns = netdev_priv(netdev);
+
+ if (!ns->phc)
+ return -EOPNOTSUPP;
+
+ switch (config->tx_type) {
+ case HWTSTAMP_TX_OFF:
+ ns->tstamp_config.tx_type = HWTSTAMP_TX_OFF;
+ break;
+ case HWTSTAMP_TX_ON:
+ ns->tstamp_config.tx_type = HWTSTAMP_TX_ON;
+ break;
+ default:
+ return -ERANGE;
+ }
+
+ switch (config->rx_filter) {
+ case HWTSTAMP_FILTER_NONE:
+ ns->tstamp_config.rx_filter = HWTSTAMP_FILTER_NONE;
+ break;
+ case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
+ case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
+ case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
+ case HWTSTAMP_FILTER_PTP_V2_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_L4_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
+ case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ:
+ case HWTSTAMP_FILTER_NTP_ALL:
+ case HWTSTAMP_FILTER_ALL:
+ ns->tstamp_config.rx_filter = HWTSTAMP_FILTER_ALL;
+ break;
+ default:
+ return -ERANGE;
+ }
+
+ return 0;
+}
+
+static int nsim_get_ts_config(struct net_device *netdev,
+ struct kernel_hwtstamp_config *config)
+{
+ struct netdevsim *ns = netdev_priv(netdev);
+
+ *config = ns->tstamp_config;
+ return 0;
+}
+
static void nsim_set_rx_mode(struct net_device *dev)
{
}
@@ -612,6 +701,8 @@ static const struct net_device_ops nsim_netdev_ops = {
.ndo_open = nsim_open,
.ndo_stop = nsim_stop,
.net_shaper_ops = &nsim_shaper_ops,
+ .ndo_hwtstamp_get = nsim_get_ts_config,
+ .ndo_hwtstamp_set = nsim_set_ts_config,
};
static const struct net_device_ops nsim_vf_netdev_ops = {
diff --git a/drivers/net/netdevsim/netdevsim.h b/drivers/net/netdevsim/netdevsim.h
index c7de53706ec4..873a81ed6dcd 100644
--- a/drivers/net/netdevsim/netdevsim.h
+++ b/drivers/net/netdevsim/netdevsim.h
@@ -103,6 +103,7 @@ struct netdevsim {
struct net_device *netdev;
struct nsim_dev *nsim_dev;
struct nsim_dev_port *nsim_dev_port;
+ struct kernel_hwtstamp_config tstamp_config;
struct mock_phc *phc;
struct nsim_rq **rq;
--
2.53.0
^ permalink raw reply related
* [PATCH v4 net-next 3/3] selftests:net: Implement ptp4l sync test using netdevsim
From: Maciek Machnikowski @ 2026-04-27 16:47 UTC (permalink / raw)
To: netdev
Cc: kuba, maciek, richardcochran, milena.olech, willemdebruijn.kernel,
andrew, vadim.fedorenko, horms
In-Reply-To: <20260427164727.15418-1-maciek@machnikowski.net>
Add PTP synchronization test using ptp4l and netdevsim.
The test creates two netdevsim adapters, links them together
and runs the ptp4l leader and ptp4l follower on two ends
of the netdevsim link and waits for the follower to report the
synchronized state (s2) in its output log.
This implementation runs the test runs over IPv4 link.
Signed-off-by: Maciek Machnikowski <maciek@machnikowski.net>
---
tools/testing/selftests/net/Makefile | 1 +
tools/testing/selftests/net/ptp.py | 184 +++++++++++++++++++++++++++
2 files changed, 185 insertions(+)
create mode 100755 tools/testing/selftests/net/ptp.py
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 231245a95879..a5fc28183896 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -69,6 +69,7 @@ TEST_PROGS := \
nl_nlctrl.py \
pmtu.sh \
psock_snd.sh \
+ ptp.py \
reuseaddr_ports_exhausted.sh \
reuseport_addr_any.sh \
route_hint.sh \
diff --git a/tools/testing/selftests/net/ptp.py b/tools/testing/selftests/net/ptp.py
new file mode 100755
index 000000000000..dd6f12cf3d91
--- /dev/null
+++ b/tools/testing/selftests/net/ptp.py
@@ -0,0 +1,184 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# By Maciek Machnikowski <maciek@machnikowski.net> (c) 2026,
+#
+# Self-tests for HW timestamping and 1588 synchronization
+#
+# This test runs a ptp4l leader/follower pair and waits for follower sync
+# state (s2), using one common execution path.
+#
+# By default, it:
+# - Creates two netdevsim instances in separate namespaces
+# - Assigns IPv4 addresses and links them together
+# - Uses those interfaces as leader/follower endpoints for ptp4l
+#
+# Optional: --leader and --follower override endpoints with existing
+# interfaces. Each argument can be either "ifname" (initial netns) or
+# "netns:ifname". Both options must be provided together.
+
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+import time
+
+from lib.py import (
+ NetNS,
+ NetdevSimDev,
+ KsftSkipEx,
+ defer,
+ ip,
+ ksft_exit,
+ ksft_pr,
+ ksft_run,
+ ksft_true,
+)
+
+PTP4L_SYNC_TIMEOUT = 40
+
+
+def _parse_interface_spec(spec):
+ """Return (netns_name_or_None, ifname). 'ns:ifname' uses that netns."""
+ if ":" in spec:
+ ns, ifname = spec.split(":", 1)
+ return ns, ifname
+ return None, spec
+
+
+def _strip_ptp_port_args():
+ """
+ Remove --leader/--follower from sys.argv. Return (leader_spec, follower_spec).
+ """
+ leader = follower = None
+ new_argv = [sys.argv[0]]
+ args = iter(sys.argv[1:])
+ for a in args:
+ if a == "--leader":
+ leader = next(args, None)
+ elif a == "--follower":
+ follower = next(args, None)
+ else:
+ new_argv.append(a)
+ sys.argv[:] = new_argv
+ return leader, follower
+
+
+def _run_ptp4l_wait_sync(leader_ifname, follower_ifname,
+ leader_ns=None, follower_ns=None):
+ leader_log_path, follower_log_path = _prepare_ptp4l_logs()
+ leader_proc, leader_log = _start_ptp4l(
+ leader_ifname, leader_log_path, leader_ns, ptp4l_params=["-4"]
+ )
+ follower_proc, follower_log = _start_ptp4l(
+ follower_ifname, follower_log_path, follower_ns, ptp4l_params=["-s", "-4"]
+ )
+ defer(lambda: _stop_ptp4l(leader_proc, leader_log))
+ defer(lambda: _stop_ptp4l(follower_proc, follower_log))
+
+ deadline = time.monotonic() + PTP4L_SYNC_TIMEOUT
+ while time.monotonic() < deadline:
+ try:
+ with open(follower_log_path) as f:
+ if " s2 " in f.read():
+ return
+ except FileNotFoundError:
+ pass
+ time.sleep(1)
+
+ ksft_pr(
+ f"ptp4l follower did not reach locked state (s2) within {PTP4L_SYNC_TIMEOUT}s"
+ )
+ try:
+ with open(follower_log_path) as f:
+ tail = f.read().strip().split("\n")[-10:]
+ ksft_pr("Follower log (last 10 lines): " + " | ".join(tail))
+ except Exception:
+ pass
+ ksft_true(False, "PTP sync timeout")
+
+
+def _start_ptp4l(ifname, log_path, ns_name, ptp4l_params=None):
+ cmd = ["ptp4l", "-i", ifname, "-m", "-P"]
+ if ptp4l_params:
+ cmd.extend(ptp4l_params)
+ if ns_name is not None:
+ cmd = ["ip", "netns", "exec", ns_name] + cmd
+
+ log_file = open(log_path, "w")
+ proc = subprocess.Popen(cmd, stdout=log_file, stderr=subprocess.STDOUT)
+ return proc, log_file
+
+
+def _stop_ptp4l(proc, log_file):
+ try:
+ proc.terminate()
+ proc.wait(timeout=5)
+ except (OSError, subprocess.TimeoutExpired):
+ try:
+ proc.kill()
+ proc.wait(timeout=2)
+ except (OSError, subprocess.TimeoutExpired):
+ pass
+ finally:
+ log_file.close()
+
+
+def _prepare_ptp4l_logs():
+ leader_log = tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".log")
+ follower_log = tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".log")
+ leader_log.close()
+ follower_log.close()
+ defer(os.unlink, leader_log.name)
+ defer(os.unlink, follower_log.name)
+ return leader_log.name, follower_log.name
+
+
+def ptp_sync_test(leader_spec=None, follower_spec=None):
+ if not shutil.which("ptp4l"):
+ raise KsftSkipEx("ptp4l command not found. Skipping PTP sync test")
+
+ use_custom = leader_spec is not None
+ if use_custom ^ (follower_spec is not None):
+ ksft_true(False, "PTP sync: specify both --leader and --follower or neither")
+ return
+
+ if use_custom:
+ leader_ns, if1 = _parse_interface_spec(leader_spec)
+ follower_ns, if2 = _parse_interface_spec(follower_spec)
+
+ _run_ptp4l_wait_sync(if1, if2, leader_ns, follower_ns)
+ return
+
+ with NetNS("nssv") as nssv, NetNS("nscl") as nscl, \
+ NetdevSimDev(port_count=1, queue_count=1, ns=nssv) as nsimdevsv, \
+ NetdevSimDev(port_count=1, queue_count=1, ns=nscl) as nsimdevcl:
+
+ nsimsv = nsimdevsv.nsims[0]
+ nsimcl = nsimdevcl.nsims[0]
+
+ ip(f"addr add 192.168.1.1/24 dev {nsimsv.ifname}", ns=nssv.name)
+ ip(f"link set dev {nsimsv.ifname} up", ns=nssv.name)
+
+ ip(f"addr add 192.168.1.2/24 dev {nsimcl.ifname}", ns=nscl.name)
+ ip(f"link set dev {nsimcl.ifname} up", ns=nscl.name)
+
+ nssv_path = f"/var/run/netns/{nssv.name}"
+ nscl_path = f"/var/run/netns/{nscl.name}"
+
+ with open(nssv_path) as nssv_file, open(nscl_path) as nscl_file:
+ link_val = f"{nssv_file.fileno()}:{nsimsv.ifindex} {nscl_file.fileno()}:{nsimcl.ifindex}"
+ NetdevSimDev.ctrl_write("link_device", link_val)
+ _run_ptp4l_wait_sync(nsimsv.ifname, nsimcl.ifname, nssv.name, nscl.name)
+ NetdevSimDev.ctrl_write("unlink_device", f"{nssv_file.fileno()}:{nsimsv.ifindex}")
+
+
+def main():
+ leader, follower = _strip_ptp_port_args()
+ ksft_run([ptp_sync_test], args=(leader, follower))
+ ksft_exit()
+
+
+if __name__ == "__main__":
+ main()
--
2.53.0
^ permalink raw reply related
* Re: [PATCH net v4 4/4] ice: skip unnecessary VF reset when setting trust
From: Simon Horman @ 2026-04-27 16:50 UTC (permalink / raw)
To: jtornosm
Cc: 'Simon Horman', netdev, intel-wired-lan,
przemyslaw.kitszel, aleksandr.loktionov, jacob.e.keller,
jesse.brandeburg, anthony.l.nguyen, davem, edumazet, kuba, pabeni
In-Reply-To: <20260423130405.139568-5-jtornosm@redhat.com>
From: 'Simon Horman' <horms@kernel.org>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ice: skip unnecessary VF reset when setting trust
This patch attempts to optimize VF trust setting changes by skipping the VF
reset when granting trust, and only resetting when revoking trust if MAC LLDP
filters are present. It also fixes a race condition by doing the reset before
clearing the LLDP filters.
> diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
> index 7e00e091756ddc..8686c382404f3f 100644
> --- a/drivers/net/ethernet/intel/ice/ice_sriov.c
> +++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
[ ... ]
> @@ -1399,11 +1427,16 @@ int ice_set_vf_trust(struct net_device *netdev, int vf_id, bool trusted)
>
> mutex_lock(&vf->cfg_lock);
>
> - while (!trusted && vf->num_mac_lldp)
> - ice_vf_update_mac_lldp_num(vf, ice_get_vf_vsi(vf), false);
> -
> + /* Reset only if revoking trust with MAC LLDP filters */
> + if (!trusted && vf->num_mac_lldp) {
> + ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
Does ice_reset_vf() clear vf->num_mac_lldp to 0 when it calls
ice_vf_pre_vsi_rebuild() internally?
If so, does this make the following while loop dead code, bypassing the
ice_vf_update_mac_lldp_num() calls that cleanly decrement the count and
trigger ice_vsi_cfg_sw_lldp() to clean up the LLDP software switch rules?
> + while (vf->num_mac_lldp)
> + ice_vf_update_mac_lldp_num(vf, ice_get_vf_vsi(vf), false);
> + } else {
> + ice_setup_vf_trust(vf, trusted);
> + }
If we revoke trust (!trusted) and vf->num_mac_lldp > 0, we enter the if block
and skip the else block containing ice_setup_vf_trust().
When ice_reset_vf() rebuilds the VSI, vf->trusted is still true. Will the
rebuild process use this true value to set the ICE_VIRTCHNL_VF_CAP_PRIVILEGE
bit in vf->vf_caps?
Since ice_setup_vf_trust() is skipped, does this fail to explicitly clear the
privilege capability bit? Could a malicious VM add a MAC LLDP filter before
trust is revoked to ensure num_mac_lldp > 0, thereby retaining its privileges?
Furthermore, when revoking trust and vf->num_mac_lldp == 0, we take the else
branch and skip ice_reset_vf().
Trusted VFs are permitted to allocate more than ICE_MAX_MACADDR_PER_VF or
ICE_MAX_VLAN_PER_VF limits. Previously, the unconditional ice_reset_vf()
would tear down the VSI and purge these extra non-default filters.
Without the reset, is there any logic in ice_setup_vf_trust() to prune the
extra MAC and VLAN filters that were added while the VF was trusted? Will
skipping the reset allow an untrusted VF to retain excessive hardware filters
indefinitely and potentially exhaust PF resources?
> vf->trusted = trusted;
> - ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
> +
> dev_info(ice_pf_to_dev(pf), "VF %u is now %strusted\n",
> vf_id, trusted ? "" : "un");
>
^ permalink raw reply
* Re: [GIT PULL] Networking deletions for 7.1
From: Pratyush Yadav @ 2026-04-27 16:56 UTC (permalink / raw)
To: Roman Gushchin
Cc: Jakub Kicinski, davem, netdev, linux-kernel, Paolo Abeni,
Linus Torvalds, Pratyush Yadav
In-Reply-To: <7ia4se8lb0vf.fsf@castle.c.googlers.com>
On Fri, Apr 24 2026, Roman Gushchin wrote:
> 2c on Sashiko:
>
> 1) I'm working on an infrastructure to separate pre-existing issues from
> new issues. My current thinking is to stop reporting these issues with
> reviews of new patches and instead put them into some database and give
> maintainers access to it. Sashiko will automatically deduplicate issues
> and index them by the source file/subsystem. Hopefully it will mean that
> maintainers will see only a limited number of issues in source files
> they support. But I have yet to see how it works in practice.
>
> But I'm somewhat concerned that this way many of these issues will
> remain there forever and by reporting them with new material we actually
> have better chances to get them fixes. Maybe it should be configurable
> per-subsystem. I'm very open for ideas here.
Yep, I agree. When I am looking at a series, the context is fresh in my
mind and if there are small fixes I can write them and send quickly. I
would be less likely to wade through a database, since it is a lot
harder to build the mental context. But then I work with subsystems that
get far fewer patches than something like networking so the noise isn't
that much of a problem for me.
Maybe you can visually separate the pre-exsiting issues (perhaps also
allow filtering?) so people can quickly see the problems with the patch
and don't always have to parse the rest?
>
> 2) Re false positives vs finding more bugs I had the same experience.
> It's easy to tweak it to be more conservative or creative, but it comes
> at a price. It seems like the real answer is simple a better model. We
> saw a big improvement internally switching from Gemini Pro 3.0 to 3.1.
>
> Thanks
--
Regards,
Pratyush Yadav
^ permalink raw reply
* [PATCH net-next] selftests: drv-net: Enable ntuple-filters if supported
From: Dimitri Daskalakis @ 2026-04-27 16:58 UTC (permalink / raw)
To: David S . Miller
Cc: Andrew Lunn, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Shuah Khan, Dimitri Daskalakis, David Wei, Joe Damato,
Dragos Tatulea, Vishwanath Seshagiri, Pavel Begunkov,
Simon Horman, Pavan Chebbi, Michael Chan, Gal Pressman, netdev
From: Dimitri Daskalakis <daskald@meta.com>
Certain devices which support ntuple-filters do not enable the feature
by default. The existing tests will skip (if they check for the feature),
or fail if they blindly attempt to install rules. Therefore, attempt to turn
on ntuple-filters if the device supports them.
Signed-off-by: Dimitri Daskalakis <daskald@meta.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
tools/testing/selftests/drivers/net/gro.py | 10 ++++++++++
tools/testing/selftests/drivers/net/hw/gro_hw.py | 10 ++++++++++
tools/testing/selftests/drivers/net/hw/iou-zcrx.py | 12 ++++++++++++
tools/testing/selftests/drivers/net/hw/ntuple.py | 5 ++++-
| 7 ++++---
5 files changed, 40 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/drivers/net/gro.py b/tools/testing/selftests/drivers/net/gro.py
index 221f27e57147..5ffaa7bdbff4 100755
--- a/tools/testing/selftests/drivers/net/gro.py
+++ b/tools/testing/selftests/drivers/net/gro.py
@@ -132,11 +132,21 @@ def _get_queue_stats(cfg, queue_id):
return {}
+def _require_ntuple(cfg):
+ features = ethtool(f"-k {cfg.ifname}", json=True)[0]
+ if not features["ntuple-filters"]["active"]:
+ if features["ntuple-filters"]["fixed"]:
+ raise KsftXfailEx("Device does not support ntuple-filters")
+ ethtool(f"-K {cfg.ifname} ntuple-filters on")
+ defer(ethtool, f"-K {cfg.ifname} ntuple-filters off")
+
+
def _setup_isolated_queue(cfg):
"""Set up an isolated queue for testing using ntuple filter.
Remove queue 1 from the default RSS context and steer test traffic to it.
"""
+ _require_ntuple(cfg)
test_queue = 1
qcnt = len(glob.glob(f"/sys/class/net/{cfg.ifname}/queues/rx-*"))
diff --git a/tools/testing/selftests/drivers/net/hw/gro_hw.py b/tools/testing/selftests/drivers/net/hw/gro_hw.py
index 10e08b22ee0e..70e76e3888bd 100755
--- a/tools/testing/selftests/drivers/net/hw/gro_hw.py
+++ b/tools/testing/selftests/drivers/net/hw/gro_hw.py
@@ -51,11 +51,21 @@ def _resolve_dmac(cfg, ipver):
return getattr(cfg, attr)
+def _require_ntuple(cfg):
+ features = ethtool(f"-k {cfg.ifname}", json=True)[0]
+ if not features["ntuple-filters"]["active"]:
+ if features["ntuple-filters"]["fixed"]:
+ raise KsftSkipEx("Device does not support ntuple-filters")
+ ethtool(f"-K {cfg.ifname} ntuple-filters on")
+ defer(ethtool, f"-K {cfg.ifname} ntuple-filters off")
+
+
def _setup_isolated_queue(cfg):
"""Set up an isolated queue for testing using ntuple filter.
Remove queue 1 from the default RSS context and steer test traffic to it.
"""
+ _require_ntuple(cfg)
test_queue = 1
qcnt = len(glob.glob(f"/sys/class/net/{cfg.ifname}/queues/rx-*"))
diff --git a/tools/testing/selftests/drivers/net/hw/iou-zcrx.py b/tools/testing/selftests/drivers/net/hw/iou-zcrx.py
index e81724cb5542..d72b76ba0835 100755
--- a/tools/testing/selftests/drivers/net/hw/iou-zcrx.py
+++ b/tools/testing/selftests/drivers/net/hw/iou-zcrx.py
@@ -100,12 +100,22 @@ def rss(cfg):
defer(ethtool, f"-N {cfg.ifname} delete {flow_rule_id}")
+def _require_ntuple(cfg):
+ features = ethtool(f"-k {cfg.ifname}", json=True)[0]
+ if not features["ntuple-filters"]["active"]:
+ if features["ntuple-filters"]["fixed"]:
+ raise KsftSkipEx("Device does not support ntuple-filters")
+ ethtool(f"-K {cfg.ifname} ntuple-filters on")
+ defer(ethtool, f"-K {cfg.ifname} ntuple-filters off")
+
+
@ksft_variants([
KsftNamedVariant("single", single),
KsftNamedVariant("rss", rss),
])
def test_zcrx(cfg, setup) -> None:
cfg.require_ipver('6')
+ _require_ntuple(cfg)
setup(cfg)
rx_cmd = f"{cfg.bin_local} -s -p {cfg.port} -i {cfg.ifname} -q {cfg.target}"
@@ -121,6 +131,7 @@ def test_zcrx(cfg, setup) -> None:
])
def test_zcrx_oneshot(cfg, setup) -> None:
cfg.require_ipver('6')
+ _require_ntuple(cfg)
setup(cfg)
rx_cmd = f"{cfg.bin_local} -s -p {cfg.port} -i {cfg.ifname} -q {cfg.target} -o 4"
@@ -134,6 +145,7 @@ def test_zcrx_large_chunks(cfg) -> None:
"""Test zcrx with large buffer chunks."""
cfg.require_ipver('6')
+ _require_ntuple(cfg)
hp_file = "/proc/sys/vm/nr_hugepages"
with open(hp_file, 'r+', encoding='utf-8') as f:
diff --git a/tools/testing/selftests/drivers/net/hw/ntuple.py b/tools/testing/selftests/drivers/net/hw/ntuple.py
index 232733142c02..ef4604bfa8ef 100755
--- a/tools/testing/selftests/drivers/net/hw/ntuple.py
+++ b/tools/testing/selftests/drivers/net/hw/ntuple.py
@@ -22,7 +22,10 @@ class NtupleField(Enum):
def _require_ntuple(cfg):
features = ethtool(f"-k {cfg.ifname}", json=True)[0]
if not features["ntuple-filters"]["active"]:
- raise KsftSkipEx("Ntuple filters not enabled on the device: " + str(features["ntuple-filters"]))
+ if features["ntuple-filters"]["fixed"]:
+ raise KsftSkipEx("Device does not support ntuple-filters")
+ ethtool(f"-K {cfg.ifname} ntuple-filters on")
+ defer(ethtool, f"-K {cfg.ifname} ntuple-filters off")
def _get_rx_cnts(cfg, prev=None):
--git a/tools/testing/selftests/drivers/net/hw/rss_ctx.py b/tools/testing/selftests/drivers/net/hw/rss_ctx.py
index 51f4e7bc3e5d..62dbc2ac7841 100755
--- a/tools/testing/selftests/drivers/net/hw/rss_ctx.py
+++ b/tools/testing/selftests/drivers/net/hw/rss_ctx.py
@@ -57,9 +57,10 @@ def ethtool_create(cfg, act, opts):
def require_ntuple(cfg):
features = ethtool(f"-k {cfg.ifname}", json=True)[0]
if not features["ntuple-filters"]["active"]:
- # ntuple is more of a capability than a config knob, don't bother
- # trying to enable it (until some driver actually needs it).
- raise KsftSkipEx("Ntuple filters not enabled on the device: " + str(features["ntuple-filters"]))
+ if features["ntuple-filters"]["fixed"]:
+ raise KsftSkipEx("Device does not support ntuple-filters")
+ ethtool(f"-K {cfg.ifname} ntuple-filters on")
+ defer(ethtool, f"-K {cfg.ifname} ntuple-filters off")
def require_context_cnt(cfg, need_cnt):
--
2.52.0
^ permalink raw reply related
* Re: [syzbot] [virt?] [net?] memory leak in __vsock_create (2)
From: syzbot @ 2026-04-27 17:03 UTC (permalink / raw)
To: davem, edumazet, horms, kuba, linux-kernel, netdev, pabeni,
sgarzare, syzkaller-bugs, virtualization
In-Reply-To: <ae-MTiL0vf-y7Ygz@sgarzare-redhat>
Hello,
syzbot has tested the proposed patch but the reproducer is still triggering an issue:
memory leak in prepare_creds
2026/04/27 17:01:37 executed programs: 5
BUG: memory leak
unreferenced object 0xffff888103b7b900 (size 184):
comm "syz-executor", pid 6458, jiffies 4294946243
hex dump (first 32 bytes):
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace (crc 5efbd4bc):
kmemleak_alloc_recursive include/linux/kmemleak.h:44 [inline]
slab_post_alloc_hook mm/slub.c:4574 [inline]
slab_alloc_node mm/slub.c:4898 [inline]
kmem_cache_alloc_noprof+0x36c/0x480 mm/slub.c:4905
prepare_creds+0x22/0x600 kernel/cred.c:185
copy_creds+0x44/0x290 kernel/cred.c:286
copy_process+0x920/0x2cf0 kernel/fork.c:2123
kernel_clone+0xde/0x700 kernel/fork.c:2723
__do_sys_clone+0x7f/0xb0 kernel/fork.c:2864
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0xee/0xf80 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: memory leak
unreferenced object 0xffff88810ad103a0 (size 32):
comm "syz-executor", pid 6458, jiffies 4294946243
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
f8 56 0a 00 81 88 ff ff 00 00 00 00 00 00 00 00 .V..............
backtrace (crc 109407f3):
kmemleak_alloc_recursive include/linux/kmemleak.h:44 [inline]
slab_post_alloc_hook mm/slub.c:4574 [inline]
slab_alloc_node mm/slub.c:4898 [inline]
__do_kmalloc_node mm/slub.c:5294 [inline]
__kmalloc_noprof+0x3b7/0x550 mm/slub.c:5307
kmalloc_noprof include/linux/slab.h:954 [inline]
kzalloc_noprof include/linux/slab.h:1188 [inline]
lsm_blob_alloc+0x4d/0x80 security/security.c:218
lsm_cred_alloc security/security.c:235 [inline]
security_prepare_creds+0x2d/0x290 security/security.c:2866
prepare_creds+0x395/0x600 kernel/cred.c:215
copy_creds+0x44/0x290 kernel/cred.c:286
copy_process+0x920/0x2cf0 kernel/fork.c:2123
kernel_clone+0xde/0x700 kernel/fork.c:2723
__do_sys_clone+0x7f/0xb0 kernel/fork.c:2864
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0xee/0xf80 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: memory leak
unreferenced object 0xffff888111516800 (size 1272):
comm "kworker/1:3", pid 5684, jiffies 4294946243
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
28 00 0b 40 00 00 00 00 00 00 00 00 00 00 00 00 (..@............
backtrace (crc 5e448183):
kmemleak_alloc_recursive include/linux/kmemleak.h:44 [inline]
slab_post_alloc_hook mm/slub.c:4574 [inline]
slab_alloc_node mm/slub.c:4898 [inline]
kmem_cache_alloc_noprof+0x36c/0x480 mm/slub.c:4905
sk_prot_alloc+0x3e/0x1b0 net/core/sock.c:2241
sk_alloc+0x36/0x460 net/core/sock.c:2303
__vsock_create.constprop.0+0x38/0x2f0 net/vmw_vsock/af_vsock.c:907
virtio_transport_recv_listen net/vmw_vsock/virtio_transport_common.c:1566 [inline]
virtio_transport_recv_pkt+0x88d/0xfb0 net/vmw_vsock/virtio_transport_common.c:1693
vsock_loopback_work+0x104/0x140 net/vmw_vsock/vsock_loopback.c:142
process_one_work+0x277/0x5b0 kernel/workqueue.c:3302
process_scheduled_works kernel/workqueue.c:3385 [inline]
worker_thread+0x255/0x4a0 kernel/workqueue.c:3466
kthread+0x14e/0x1a0 kernel/kthread.c:436
ret_from_fork+0x219/0x490 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
BUG: memory leak
unreferenced object 0xffff88812daab5e0 (size 32):
comm "kworker/1:3", pid 5684, jiffies 4294946243
hex dump (first 32 bytes):
f8 56 0a 00 81 88 ff ff 00 00 00 00 00 00 00 00 .V..............
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace (crc 79381f4a):
kmemleak_alloc_recursive include/linux/kmemleak.h:44 [inline]
slab_post_alloc_hook mm/slub.c:4574 [inline]
slab_alloc_node mm/slub.c:4898 [inline]
__do_kmalloc_node mm/slub.c:5294 [inline]
__kmalloc_noprof+0x3b7/0x550 mm/slub.c:5307
kmalloc_noprof include/linux/slab.h:954 [inline]
kzalloc_noprof include/linux/slab.h:1188 [inline]
lsm_blob_alloc+0x4d/0x80 security/security.c:218
lsm_sock_alloc security/security.c:4478 [inline]
security_sk_alloc+0x2d/0x290 security/security.c:4494
sk_prot_alloc+0x8f/0x1b0 net/core/sock.c:2250
sk_alloc+0x36/0x460 net/core/sock.c:2303
__vsock_create.constprop.0+0x38/0x2f0 net/vmw_vsock/af_vsock.c:907
virtio_transport_recv_listen net/vmw_vsock/virtio_transport_common.c:1566 [inline]
virtio_transport_recv_pkt+0x88d/0xfb0 net/vmw_vsock/virtio_transport_common.c:1693
vsock_loopback_work+0x104/0x140 net/vmw_vsock/vsock_loopback.c:142
process_one_work+0x277/0x5b0 kernel/workqueue.c:3302
process_scheduled_works kernel/workqueue.c:3385 [inline]
worker_thread+0x255/0x4a0 kernel/workqueue.c:3466
kthread+0x14e/0x1a0 kernel/kthread.c:436
ret_from_fork+0x219/0x490 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
BUG: memory leak
unreferenced object 0xffff88810b1a75a0 (size 96):
comm "kworker/1:3", pid 5684, jiffies 4294946243
hex dump (first 32 bytes):
00 68 51 11 81 88 ff ff 00 00 00 00 00 00 00 00 .hQ.............
00 00 00 00 00 00 00 00 00 00 04 00 00 00 00 00 ................
backtrace (crc 428f2031):
kmemleak_alloc_recursive include/linux/kmemleak.h:44 [inline]
slab_post_alloc_hook mm/slub.c:4574 [inline]
slab_alloc_node mm/slub.c:4898 [inline]
__kmalloc_cache_noprof+0x371/0x480 mm/slub.c:5410
kmalloc_noprof include/linux/slab.h:950 [inline]
kzalloc_noprof include/linux/slab.h:1188 [inline]
virtio_transport_do_socket_init+0x2b/0xf0 net/vmw_vsock/virtio_transport_common.c:925
vsock_assign_transport+0x3a3/0x460 net/vmw_vsock/af_vsock.c:656
virtio_transport_recv_listen net/vmw_vsock/virtio_transport_common.c:1582 [inline]
virtio_transport_recv_pkt+0x8e5/0xfb0 net/vmw_vsock/virtio_transport_common.c:1693
vsock_loopback_work+0x104/0x140 net/vmw_vsock/vsock_loopback.c:142
process_one_work+0x277/0x5b0 kernel/workqueue.c:3302
process_scheduled_works kernel/workqueue.c:3385 [inline]
worker_thread+0x255/0x4a0 kernel/workqueue.c:3466
kthread+0x14e/0x1a0 kernel/kthread.c:436
ret_from_fork+0x219/0x490 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
BUG: memory leak
unreferenced object 0xffff888111516300 (size 1272):
comm "kworker/1:3", pid 5684, jiffies 4294946244
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
28 00 0b 40 00 00 00 00 00 00 00 00 00 00 00 00 (..@............
backtrace (crc e1cd45d1):
kmemleak_alloc_recursive include/linux/kmemleak.h:44 [inline]
slab_post_alloc_hook mm/slub.c:4574 [inline]
slab_alloc_node mm/slub.c:4898 [inline]
kmem_cache_alloc_noprof+0x36c/0x480 mm/slub.c:4905
sk_prot_alloc+0x3e/0x1b0 net/core/sock.c:2241
sk_alloc+0x36/0x460 net/core/sock.c:2303
__vsock_create.constprop.0+0x38/0x2f0 net/vmw_vsock/af_vsock.c:907
virtio_transport_recv_listen net/vmw_vsock/virtio_transport_common.c:1566 [inline]
virtio_transport_recv_pkt+0x88d/0xfb0 net/vmw_vsock/virtio_transport_common.c:1693
vsock_loopback_work+0x104/0x140 net/vmw_vsock/vsock_loopback.c:142
process_one_work+0x277/0x5b0 kernel/workqueue.c:3302
process_scheduled_works kernel/workqueue.c:3385 [inline]
worker_thread+0x255/0x4a0 kernel/workqueue.c:3466
kthread+0x14e/0x1a0 kernel/kthread.c:436
ret_from_fork+0x219/0x490 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
connection error: failed to recv *flatrpc.ExecutorMessageRawT: EOF
Tested on:
commit: 39ff9a4a vsock/virtio: fix socket leak on close_timeou..
git tree: https://github.com/stefano-garzarella/linux.git fix-syzbot-memleak-vsock-create
console output: https://syzkaller.appspot.com/x/log.txt?x=1742b896580000
kernel config: https://syzkaller.appspot.com/x/.config?x=dfcc8f993a958a78
dashboard link: https://syzkaller.appspot.com/bug?extid=1b2c9c4a0f8708082678
compiler: gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44
Note: no patches were applied.
^ permalink raw reply
* [RFC PATCH v1 0/9] uaccess: Convert small fixed size copy_{to/from}_user() to scoped user access
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
A lot of copy_from_user() and copy_to_user() perform copies of small
fixed size pieces of data between kernel and userspace, and don't
care about partial copies.
copy_from_user() and copy_to_user() are big functions optimised for
copying large amount of data, with cache management, etc ... This is
often overkill for small copies that could just be inlined instead.
What makes things a bit more tricky is that those copy functions
are designed to handle partial copies in case of page fault. But among
the 6000 callers of those functions, only 2% really care about the
quantity of no-copied data that those functions return. All other ones
fails as soon as the returned value is not 0, returning -EACCESS.
So first step in this series is to introduce variants called
copy_from_user_partial() and copy_to_user_partial() which will be
called by the 2% users that care about the partial copy, then the
original copy_from_user() and copy_to_user() are changed to return
-EFAULT when the copy fails.
Then the second step is to implement copy of small fixed-size data
with scoped user access instead of calling the arch specific heavy
user copy functions.
Patch 5, can be split in different patches for each archicture or
subsystem, but let's get a first feedback and agree on the principle.
Christophe Leroy (CS GROUP) (9):
uaccess: Split check_zeroed_user() out of usercopy.c
uaccess: Convert INLINE_COPY_{TO/FROM}_USER to kconfig and reduce
ifdefery
x86/umip: Be stricter in fixup_umip_exception()
uaccess: Introduce copy_{to/from}_user_partial()
uaccess: Switch to copy_{to/from}_user_partial() when relevant
uaccess: Change copy_{to/from}_user to return -EFAULT
x86: Add unsafe_copy_from_user()
arm64: Add unsafe_copy_from_user()
uaccess: Convert small fixed size copy_{to/from}_user() to scoped user
access
arch/alpha/Kconfig | 1 +
arch/alpha/kernel/osf_sys.c | 4 +-
arch/alpha/kernel/termios.c | 2 +-
arch/arc/include/asm/uaccess.h | 3 -
arch/arc/kernel/disasm.c | 2 +-
arch/arm/include/asm/uaccess.h | 2 -
arch/arm64/include/asm/gcs.h | 2 +-
arch/arm64/include/asm/uaccess.h | 30 +++--
arch/arm64/kernel/signal32.c | 2 +-
arch/csky/Kconfig | 1 +
arch/hexagon/include/asm/uaccess.h | 3 -
arch/loongarch/include/asm/uaccess.h | 3 -
arch/m68k/include/asm/uaccess.h | 3 -
arch/microblaze/include/asm/uaccess.h | 2 -
arch/mips/include/asm/uaccess.h | 3 -
arch/mips/kernel/rtlx.c | 8 +-
arch/mips/kernel/vpe.c | 2 +-
arch/nios2/include/asm/uaccess.h | 2 -
arch/openrisc/include/asm/uaccess.h | 2 -
arch/parisc/include/asm/uaccess.h | 3 -
arch/powerpc/Kconfig | 1 +
arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +-
arch/powerpc/kvm/book3s_64_mmu_radix.c | 4 +-
arch/powerpc/kvm/book3s_hv.c | 2 +-
arch/riscv/Kconfig | 1 +
arch/riscv/kernel/signal.c | 2 +-
arch/s390/include/asm/idals.h | 8 +-
arch/s390/include/asm/uaccess.h | 3 -
arch/sh/include/asm/uaccess.h | 2 -
arch/sparc/include/asm/uaccess_32.h | 3 -
arch/sparc/include/asm/uaccess_64.h | 2 -
arch/sparc/kernel/termios.c | 2 +-
arch/um/include/asm/uaccess.h | 3 -
arch/um/kernel/process.c | 2 +-
arch/x86/Kconfig | 1 +
arch/x86/include/asm/uaccess.h | 29 ++++-
arch/x86/kernel/umip.c | 2 +-
arch/x86/lib/insn-eval.c | 2 +-
arch/x86/um/signal.c | 2 +-
arch/xtensa/include/asm/uaccess.h | 2 -
drivers/android/binder_alloc.c | 2 +-
drivers/comedi/comedi_fops.c | 4 +-
drivers/dma/idxd/cdev.c | 2 +-
drivers/firmware/efi/test/efi_test.c | 2 +-
drivers/fsi/fsi-scom.c | 2 +-
.../amd/display/amdgpu_dm/amdgpu_dm_debugfs.c | 2 +-
drivers/gpu/drm/i915/gt/intel_sseu.c | 4 +-
drivers/gpu/drm/i915/i915_gem.c | 4 +-
drivers/hwtracing/intel_th/msu.c | 2 +-
drivers/misc/ibmvmc.c | 2 +-
drivers/misc/vmw_vmci/vmci_host.c | 2 +-
drivers/most/most_cdev.c | 2 +-
drivers/net/ieee802154/ca8210.c | 4 +-
drivers/net/wireless/ath/wil6210/debugfs.c | 2 +-
.../intel/iwlwifi/pcie/gen1_2/trans.c | 2 +-
drivers/net/wireless/ti/wlcore/debugfs.c | 2 +-
drivers/ps3/ps3-lpm.c | 2 +-
drivers/s390/crypto/zcrypt_api.h | 4 +-
drivers/spi/spidev.c | 2 +-
.../staging/media/atomisp/pci/atomisp_cmd.c | 8 +-
drivers/tty/tty_ioctl.c | 14 +--
drivers/tty/vt/vc_screen.c | 4 +-
drivers/usb/gadget/function/f_hid.c | 4 +-
drivers/usb/gadget/function/f_printer.c | 2 +-
drivers/vfio/vfio_iommu_type1.c | 4 +-
drivers/xen/xenbus/xenbus_dev_frontend.c | 2 +-
fs/namespace.c | 2 +-
fs/ocfs2/dlmfs/dlmfs.c | 2 +-
fs/proc/base.c | 4 +-
include/asm-generic/uaccess.h | 2 -
include/linux/bpfptr.h | 2 +-
include/linux/sockptr.h | 4 +-
include/linux/uaccess.h | 107 ++++++++++++++----
ipc/msg.c | 8 +-
ipc/sem.c | 8 +-
ipc/shm.c | 18 +--
kernel/regset.c | 2 +-
kernel/sys.c | 4 +-
lib/Kconfig | 3 +
lib/Makefile | 4 +-
lib/kfifo.c | 8 +-
lib/{usercopy.c => usercheck.c} | 22 ----
lib/usercopy.c | 66 -----------
mm/kasan/kasan_test_c.c | 4 +-
mm/memory.c | 2 +-
net/x25/af_x25.c | 2 +-
rust/helpers/uaccess.c | 6 +-
sound/pci/emu10k1/emufx.c | 4 +-
sound/pci/rme9652/hdsp.c | 6 +-
sound/soc/intel/avs/probes.c | 6 +-
sound/soc/sof/compress.c | 12 +-
sound/soc/sof/sof-client-probes.c | 6 +-
92 files changed, 269 insertions(+), 288 deletions(-)
copy lib/{usercopy.c => usercheck.c} (73%)
--
2.49.0
^ permalink raw reply
* [RFC PATCH v1 1/9] uaccess: Split check_zeroed_user() out of usercopy.c
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
Until commit f5a1a536fa14 ("lib: introduce copy_struct_from_user()
helper"), lib/usercopy.c was containing only the out-line version
of user copy fonctions.
That commit added function check_zeroed_user() into the same file.
Move that function into a new file named usercheck.c, so that next
patch can change usercopy.c build to a conditional build.
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
lib/Makefile | 1 +
lib/{usercopy.c => usercheck.c} | 22 ------------
lib/usercopy.c | 62 ---------------------------------
3 files changed, 1 insertion(+), 84 deletions(-)
copy lib/{usercopy.c => usercheck.c} (73%)
diff --git a/lib/Makefile b/lib/Makefile
index f33a24bf1c19..7c0334d7675b 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -59,6 +59,7 @@ obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \
percpu-refcount.o rhashtable.o base64.o \
once.o refcount.o rcuref.o usercopy.o errseq.o bucket_locks.o \
generic-radix-tree.o bitmap-str.o
+obj-y += usercheck.o
obj-y += string_helpers.o
obj-y += hexdump.o
obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
diff --git a/lib/usercopy.c b/lib/usercheck.c
similarity index 73%
copy from lib/usercopy.c
copy to lib/usercheck.c
index b00a3a957de6..15b0d9a18435 100644
--- a/lib/usercopy.c
+++ b/lib/usercheck.c
@@ -2,32 +2,10 @@
#include <linux/compiler.h>
#include <linux/errno.h>
#include <linux/export.h>
-#include <linux/fault-inject-usercopy.h>
-#include <linux/instrumented.h>
#include <linux/kernel.h>
-#include <linux/nospec.h>
-#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/wordpart.h>
-/* out-of-line parts */
-
-#if !defined(INLINE_COPY_FROM_USER)
-unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
-{
- return _inline_copy_from_user(to, from, n);
-}
-EXPORT_SYMBOL(_copy_from_user);
-#endif
-
-#if !defined(INLINE_COPY_TO_USER)
-unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
-{
- return _inline_copy_to_user(to, from, n);
-}
-EXPORT_SYMBOL(_copy_to_user);
-#endif
-
/**
* check_zeroed_user: check if a userspace buffer only contains zero bytes
* @from: Source address, in userspace.
diff --git a/lib/usercopy.c b/lib/usercopy.c
index b00a3a957de6..7a93f56d81dd 100644
--- a/lib/usercopy.c
+++ b/lib/usercopy.c
@@ -1,14 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
-#include <linux/compiler.h>
-#include <linux/errno.h>
#include <linux/export.h>
-#include <linux/fault-inject-usercopy.h>
-#include <linux/instrumented.h>
-#include <linux/kernel.h>
-#include <linux/nospec.h>
-#include <linux/string.h>
#include <linux/uaccess.h>
-#include <linux/wordpart.h>
/* out-of-line parts */
@@ -27,57 +19,3 @@ unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
}
EXPORT_SYMBOL(_copy_to_user);
#endif
-
-/**
- * check_zeroed_user: check if a userspace buffer only contains zero bytes
- * @from: Source address, in userspace.
- * @size: Size of buffer.
- *
- * This is effectively shorthand for "memchr_inv(from, 0, size) == NULL" for
- * userspace addresses (and is more efficient because we don't care where the
- * first non-zero byte is).
- *
- * Returns:
- * * 0: There were non-zero bytes present in the buffer.
- * * 1: The buffer was full of zero bytes.
- * * -EFAULT: access to userspace failed.
- */
-int check_zeroed_user(const void __user *from, size_t size)
-{
- unsigned long val;
- uintptr_t align = (uintptr_t) from % sizeof(unsigned long);
-
- if (unlikely(size == 0))
- return 1;
-
- from -= align;
- size += align;
-
- if (!user_read_access_begin(from, size))
- return -EFAULT;
-
- unsafe_get_user(val, (unsigned long __user *) from, err_fault);
- if (align)
- val &= ~aligned_byte_mask(align);
-
- while (size > sizeof(unsigned long)) {
- if (unlikely(val))
- goto done;
-
- from += sizeof(unsigned long);
- size -= sizeof(unsigned long);
-
- unsafe_get_user(val, (unsigned long __user *) from, err_fault);
- }
-
- if (size < sizeof(unsigned long))
- val &= aligned_byte_mask(size);
-
-done:
- user_read_access_end();
- return (val == 0);
-err_fault:
- user_read_access_end();
- return -EFAULT;
-}
-EXPORT_SYMBOL(check_zeroed_user);
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 2/9] uaccess: Convert INLINE_COPY_{TO/FROM}_USER to kconfig and reduce ifdefery
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
Among the 21 architectures supported by the kernel, 16 define both
INLINE_COPY_TO_USER and INLINE_COPY_FROM_USER while the 5 other ones
don't define any of the two.
To simplify and reduce risk of mistakes, convert them to a single
kconfig item named CONFIG_ARCH_WANTS_NOINLINE_COPY which will be
selected by the 5 architectures that don't want inlined copy.
To minimise complication in a later patch, also remove
ifdefery and replace it with IS_ENABLED().
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
arch/alpha/Kconfig | 1 +
arch/arc/include/asm/uaccess.h | 3 ---
arch/arm/include/asm/uaccess.h | 2 --
arch/arm64/include/asm/uaccess.h | 3 ---
arch/csky/Kconfig | 1 +
arch/hexagon/include/asm/uaccess.h | 3 ---
arch/loongarch/include/asm/uaccess.h | 3 ---
arch/m68k/include/asm/uaccess.h | 3 ---
arch/microblaze/include/asm/uaccess.h | 2 --
arch/mips/include/asm/uaccess.h | 3 ---
arch/nios2/include/asm/uaccess.h | 2 --
arch/openrisc/include/asm/uaccess.h | 2 --
arch/parisc/include/asm/uaccess.h | 3 ---
arch/powerpc/Kconfig | 1 +
arch/riscv/Kconfig | 1 +
arch/s390/include/asm/uaccess.h | 3 ---
arch/sh/include/asm/uaccess.h | 2 --
arch/sparc/include/asm/uaccess_32.h | 3 ---
arch/sparc/include/asm/uaccess_64.h | 2 --
arch/um/include/asm/uaccess.h | 3 ---
arch/x86/Kconfig | 1 +
arch/xtensa/include/asm/uaccess.h | 2 --
include/asm-generic/uaccess.h | 2 --
include/linux/uaccess.h | 32 ++++++++++++---------------
lib/Kconfig | 3 +++
lib/Makefile | 3 ++-
lib/usercopy.c | 4 ----
rust/helpers/uaccess.c | 2 +-
28 files changed, 25 insertions(+), 70 deletions(-)
diff --git a/arch/alpha/Kconfig b/arch/alpha/Kconfig
index 7b7dafe7d9df..65e533cead6b 100644
--- a/arch/alpha/Kconfig
+++ b/arch/alpha/Kconfig
@@ -11,6 +11,7 @@ config ALPHA
select ARCH_NO_PREEMPT
select ARCH_NO_SG_CHAIN
select ARCH_USE_CMPXCHG_LOCKREF
+ select ARCH_WANTS_NOINLINE_COPY_USER
select FORCE_PCI
select PCI_DOMAINS if PCI
select PCI_SYSCALL if PCI
diff --git a/arch/arc/include/asm/uaccess.h b/arch/arc/include/asm/uaccess.h
index 1e8809ea000a..e8b161b37a03 100644
--- a/arch/arc/include/asm/uaccess.h
+++ b/arch/arc/include/asm/uaccess.h
@@ -628,9 +628,6 @@ static inline unsigned long __clear_user(void __user *to, unsigned long n)
return res;
}
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
-
#define __clear_user __clear_user
#include <asm-generic/uaccess.h>
diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h
index d6ae80b5df36..7280c162bb71 100644
--- a/arch/arm/include/asm/uaccess.h
+++ b/arch/arm/include/asm/uaccess.h
@@ -616,8 +616,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
}
#define __clear_user(addr, n) (memset((void __force *)addr, 0, n), 0)
#endif
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
static inline unsigned long __must_check clear_user(void __user *to, unsigned long n)
{
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index b0c83a08dda9..1e20ec91b56f 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -456,9 +456,6 @@ do { \
unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \
} while (0)
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
-
extern unsigned long __must_check __arch_clear_user(void __user *to, unsigned long n);
static inline unsigned long __must_check __clear_user(void __user *to, unsigned long n)
{
diff --git a/arch/csky/Kconfig b/arch/csky/Kconfig
index 4331313a42ff..d010d7eb47bf 100644
--- a/arch/csky/Kconfig
+++ b/arch/csky/Kconfig
@@ -40,6 +40,7 @@ config CSKY
select ARCH_NEED_CMPXCHG_1_EMU
select ARCH_WANT_FRAME_POINTERS if !CPU_CK610 && $(cc-option,-mbacktrace)
select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT
+ select ARCH_WANTS_NOINLINE_COPY_USER
select COMMON_CLK
select CLKSRC_MMIO
select CSKY_MPINTC if CPU_CK860
diff --git a/arch/hexagon/include/asm/uaccess.h b/arch/hexagon/include/asm/uaccess.h
index bff77efc0d9a..4bf863217636 100644
--- a/arch/hexagon/include/asm/uaccess.h
+++ b/arch/hexagon/include/asm/uaccess.h
@@ -26,9 +26,6 @@ unsigned long raw_copy_from_user(void *to, const void __user *from,
unsigned long n);
unsigned long raw_copy_to_user(void __user *to, const void *from,
unsigned long n);
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
-
__kernel_size_t __clear_user_hexagon(void __user *dest, unsigned long count);
#define __clear_user(a, s) __clear_user_hexagon((a), (s))
diff --git a/arch/loongarch/include/asm/uaccess.h b/arch/loongarch/include/asm/uaccess.h
index 438269313e78..72a04ac88549 100644
--- a/arch/loongarch/include/asm/uaccess.h
+++ b/arch/loongarch/include/asm/uaccess.h
@@ -292,9 +292,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
return __copy_user((__force void *)to, from, n);
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
-
/*
* __clear_user: - Zero a block of memory in user space, with less checking.
* @addr: Destination address, in user space.
diff --git a/arch/m68k/include/asm/uaccess.h b/arch/m68k/include/asm/uaccess.h
index 64914872a5c9..20e249a6ad07 100644
--- a/arch/m68k/include/asm/uaccess.h
+++ b/arch/m68k/include/asm/uaccess.h
@@ -377,9 +377,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
return __constant_copy_to_user(to, from, n);
return __generic_copy_to_user(to, from, n);
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
-
#define __get_kernel_nofault(dst, src, type, err_label) \
do { \
type *__gk_dst = (type *)(dst); \
diff --git a/arch/microblaze/include/asm/uaccess.h b/arch/microblaze/include/asm/uaccess.h
index 3aab2f17e046..3355f541e12a 100644
--- a/arch/microblaze/include/asm/uaccess.h
+++ b/arch/microblaze/include/asm/uaccess.h
@@ -250,8 +250,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
{
return __copy_tofrom_user(to, (__force const void __user *)from, n);
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
/*
* Copy a null terminated string from userspace.
diff --git a/arch/mips/include/asm/uaccess.h b/arch/mips/include/asm/uaccess.h
index c0cede273c7c..8714caefbac8 100644
--- a/arch/mips/include/asm/uaccess.h
+++ b/arch/mips/include/asm/uaccess.h
@@ -433,9 +433,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
return __cu_len_r;
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
-
extern __kernel_size_t __bzero(void __user *addr, __kernel_size_t size);
/*
diff --git a/arch/nios2/include/asm/uaccess.h b/arch/nios2/include/asm/uaccess.h
index 6ccc9a232c23..46d7312a1c96 100644
--- a/arch/nios2/include/asm/uaccess.h
+++ b/arch/nios2/include/asm/uaccess.h
@@ -57,8 +57,6 @@ extern unsigned long
raw_copy_from_user(void *to, const void __user *from, unsigned long n);
extern unsigned long
raw_copy_to_user(void __user *to, const void *from, unsigned long n);
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
extern long strncpy_from_user(char *__to, const char __user *__from,
long __len);
diff --git a/arch/openrisc/include/asm/uaccess.h b/arch/openrisc/include/asm/uaccess.h
index d6500a374e18..c84effde867a 100644
--- a/arch/openrisc/include/asm/uaccess.h
+++ b/arch/openrisc/include/asm/uaccess.h
@@ -218,8 +218,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long size)
{
return __copy_tofrom_user((__force void *)to, from, size);
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
extern unsigned long __clear_user(void __user *addr, unsigned long size);
diff --git a/arch/parisc/include/asm/uaccess.h b/arch/parisc/include/asm/uaccess.h
index 6c531d2c847e..1dd6a1dd653f 100644
--- a/arch/parisc/include/asm/uaccess.h
+++ b/arch/parisc/include/asm/uaccess.h
@@ -197,7 +197,4 @@ unsigned long __must_check raw_copy_to_user(void __user *dst, const void *src,
unsigned long len);
unsigned long __must_check raw_copy_from_user(void *dst, const void __user *src,
unsigned long len);
-#define INLINE_COPY_TO_USER
-#define INLINE_COPY_FROM_USER
-
#endif /* __PARISC_UACCESS_H */
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index e93df95b79e7..6816f402fe3d 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -187,6 +187,7 @@ config PPC
select ARCH_WANT_LD_ORPHAN_WARN
select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP if PPC_RADIX_MMU
select ARCH_WANTS_MODULES_DATA_IN_VMALLOC if PPC_BOOK3S_32 || PPC_8xx
+ select ARCH_WANTS_NOINLINE_COPY_USER
select ARCH_WEAK_RELEASE_ACQUIRE
select AUDIT_ARCH_COMPAT_GENERIC
select BINFMT_ELF
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index d235396c4514..492b920c1a51 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -88,6 +88,7 @@ config RISCV
select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP
select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP
select ARCH_WANTS_NO_INSTR
+ select ARCH_WANTS_NOINLINE_COPY_USER if MMU
select ARCH_WANTS_THP_SWAP if HAVE_ARCH_TRANSPARENT_HUGEPAGE
select ARCH_WEAK_RELEASE_ACQUIRE if ARCH_USE_QUEUED_SPINLOCKS
select BINFMT_FLAT_NO_DATA_START_OFFSET if !MMU
diff --git a/arch/s390/include/asm/uaccess.h b/arch/s390/include/asm/uaccess.h
index dff035372601..2e0472c20da0 100644
--- a/arch/s390/include/asm/uaccess.h
+++ b/arch/s390/include/asm/uaccess.h
@@ -30,9 +30,6 @@ void debug_user_asce(int exit);
#define uaccess_kmsan_or_inline __always_inline
#endif
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
-
static uaccess_kmsan_or_inline __must_check unsigned long
raw_copy_from_user(void *to, const void __user *from, unsigned long size)
{
diff --git a/arch/sh/include/asm/uaccess.h b/arch/sh/include/asm/uaccess.h
index a79609eb14be..0cd75308e6d3 100644
--- a/arch/sh/include/asm/uaccess.h
+++ b/arch/sh/include/asm/uaccess.h
@@ -95,8 +95,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
{
return __copy_user((__force void *)to, from, n);
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
/*
* Clear the area and return remaining number of bytes
diff --git a/arch/sparc/include/asm/uaccess_32.h b/arch/sparc/include/asm/uaccess_32.h
index 43284b6ec46a..e01f43c6421c 100644
--- a/arch/sparc/include/asm/uaccess_32.h
+++ b/arch/sparc/include/asm/uaccess_32.h
@@ -190,9 +190,6 @@ static inline unsigned long raw_copy_from_user(void *to, const void __user *from
return __copy_user((__force void __user *) to, from, n);
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
-
static inline unsigned long __clear_user(void __user *addr, unsigned long size)
{
unsigned long ret;
diff --git a/arch/sparc/include/asm/uaccess_64.h b/arch/sparc/include/asm/uaccess_64.h
index b825a5dd0210..62ee0b074fec 100644
--- a/arch/sparc/include/asm/uaccess_64.h
+++ b/arch/sparc/include/asm/uaccess_64.h
@@ -231,8 +231,6 @@ unsigned long __must_check raw_copy_from_user(void *to,
unsigned long __must_check raw_copy_to_user(void __user *to,
const void *from,
unsigned long size);
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
unsigned long __must_check raw_copy_in_user(void __user *to,
const void __user *from,
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index 0df9ea4abda8..1e14260c7f0f 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -27,9 +27,6 @@ static inline int __access_ok(const void __user *ptr, unsigned long size);
#define __access_ok __access_ok
#define __clear_user __clear_user
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
-
#include <asm-generic/uaccess.h>
static inline int __access_ok(const void __user *ptr, unsigned long size)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f3f7cb01d69d..c1e58d8c6864 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -143,6 +143,7 @@ config X86
select ARCH_WANTS_CLOCKSOURCE_READ_INLINE if X86_64
select ARCH_WANTS_DYNAMIC_TASK_STRUCT
select ARCH_WANTS_NO_INSTR
+ select ARCH_WANTS_NOINLINE_COPY_USER
select ARCH_WANT_GENERAL_HUGETLB
select ARCH_WANT_HUGE_PMD_SHARE if X86_64
select ARCH_WANT_LD_ORPHAN_WARN
diff --git a/arch/xtensa/include/asm/uaccess.h b/arch/xtensa/include/asm/uaccess.h
index 56aec6d504fe..f9e1623a7be9 100644
--- a/arch/xtensa/include/asm/uaccess.h
+++ b/arch/xtensa/include/asm/uaccess.h
@@ -237,8 +237,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
prefetch(from);
return __xtensa_copy_user((__force void *)to, from, n);
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
/*
* We need to return the number of bytes not cleared. Our memset()
diff --git a/include/asm-generic/uaccess.h b/include/asm-generic/uaccess.h
index b276f783494c..fb33a71fd24e 100644
--- a/include/asm-generic/uaccess.h
+++ b/include/asm-generic/uaccess.h
@@ -91,8 +91,6 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)
memcpy((void __force *)to, from, n);
return 0;
}
-#define INLINE_COPY_FROM_USER
-#define INLINE_COPY_TO_USER
#endif /* CONFIG_UACCESS_MEMCPY */
/*
diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index 56328601218c..bd1201c81d94 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -83,8 +83,8 @@
* with that. They should not be used directly; they are used to implement
* the 6 functions (copy_{to,from}_user(), __copy_{to,from}_user_inatomic())
* that are used instead. Out of those, __... ones are inlined. Plain
- * copy_{to,from}_user() might or might not be inlined. If you want them
- * inlined, have asm/uaccess.h define INLINE_COPY_{TO,FROM}_USER.
+ * copy_{to,from}_user() might or might not be inlined. If you don't want them
+ * inlined, select CONFIG_ARCH_WANTS_NOINLINE_COPY_USER.
*
* NOTE: only copy_from_user() zero-pads the destination in case of short copy.
* Neither __copy_from_user() nor __copy_from_user_inatomic() zero anything
@@ -157,8 +157,8 @@ __copy_to_user(void __user *to, const void *from, unsigned long n)
}
/*
- * Architectures that #define INLINE_COPY_TO_USER use this function
- * directly in the normal copy_to/from_user(), the other ones go
+ * Architectures that don't select CONFIG_ARCH_WANTS_NOINLINE_COPY_USER use
+ * this function directly in the normal copy_to/from_user(), the other ones go
* through an extern _copy_to/from_user(), which expands the same code
* here.
*/
@@ -190,10 +190,9 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n)
memset(to + (n - res), 0, res);
return res;
}
-#ifndef INLINE_COPY_FROM_USER
+
extern __must_check unsigned long
_copy_from_user(void *, const void __user *, unsigned long);
-#endif
static inline __must_check unsigned long
_inline_copy_to_user(void __user *to, const void *from, unsigned long n)
@@ -207,21 +206,19 @@ _inline_copy_to_user(void __user *to, const void *from, unsigned long n)
}
return n;
}
-#ifndef INLINE_COPY_TO_USER
+
extern __must_check unsigned long
_copy_to_user(void __user *, const void *, unsigned long);
-#endif
static __always_inline unsigned long __must_check
copy_from_user(void *to, const void __user *from, unsigned long n)
{
if (!check_copy_size(to, n, false))
return n;
-#ifdef INLINE_COPY_FROM_USER
- return _inline_copy_from_user(to, from, n);
-#else
- return _copy_from_user(to, from, n);
-#endif
+ if (IS_ENABLED(ARCH_WANTS_NOINLINE_COPY_USER))
+ return _copy_from_user(to, from, n);
+ else
+ return _inline_copy_from_user(to, from, n);
}
static __always_inline unsigned long __must_check
@@ -230,11 +227,10 @@ copy_to_user(void __user *to, const void *from, unsigned long n)
if (!check_copy_size(from, n, true))
return n;
-#ifdef INLINE_COPY_TO_USER
- return _inline_copy_to_user(to, from, n);
-#else
- return _copy_to_user(to, from, n);
-#endif
+ if (IS_ENABLED(ARCH_WANTS_NOINLINE_COPY_USER))
+ return _copy_to_user(to, from, n);
+ else
+ return _inline_copy_to_user(to, from, n);
}
#ifndef copy_mc_to_kernel
diff --git a/lib/Kconfig b/lib/Kconfig
index 00a9509636c1..a2e07d4dd2bf 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -68,6 +68,9 @@ config ARCH_HAS_STRNCPY_FROM_USER
config ARCH_HAS_STRNLEN_USER
bool
+config ARCH_WANTS_NOINLINE_COPY_USER
+ bool
+
config GENERIC_STRNCPY_FROM_USER
def_bool !ARCH_HAS_STRNCPY_FROM_USER
diff --git a/lib/Makefile b/lib/Makefile
index 7c0334d7675b..f4d577910671 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -57,9 +57,10 @@ obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \
list_sort.o uuid.o iov_iter.o clz_ctz.o \
bsearch.o find_bit.o llist.o lwq.o memweight.o kfifo.o \
percpu-refcount.o rhashtable.o base64.o \
- once.o refcount.o rcuref.o usercopy.o errseq.o bucket_locks.o \
+ once.o refcount.o rcuref.o errseq.o bucket_locks.o \
generic-radix-tree.o bitmap-str.o
obj-y += usercheck.o
+obj-$(CONFIG_ARCH_WANTS_NOINLINE_COPY_USER) += usercopy.o
obj-y += string_helpers.o
obj-y += hexdump.o
obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
diff --git a/lib/usercopy.c b/lib/usercopy.c
index 7a93f56d81dd..d2deb4b0a3c5 100644
--- a/lib/usercopy.c
+++ b/lib/usercopy.c
@@ -4,18 +4,14 @@
/* out-of-line parts */
-#if !defined(INLINE_COPY_FROM_USER)
unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
{
return _inline_copy_from_user(to, from, n);
}
EXPORT_SYMBOL(_copy_from_user);
-#endif
-#if !defined(INLINE_COPY_TO_USER)
unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
{
return _inline_copy_to_user(to, from, n);
}
EXPORT_SYMBOL(_copy_to_user);
-#endif
diff --git a/rust/helpers/uaccess.c b/rust/helpers/uaccess.c
index d9625b9ee046..01de4fbbcc84 100644
--- a/rust/helpers/uaccess.c
+++ b/rust/helpers/uaccess.c
@@ -14,7 +14,7 @@ rust_helper_copy_to_user(void __user *to, const void *from, unsigned long n)
return copy_to_user(to, from, n);
}
-#ifdef INLINE_COPY_FROM_USER
+#ifndef CONFIG_ARCH_WANTS_NOINLINE_COPY_USER
__rust_helper
unsigned long rust_helper__copy_from_user(void *to, const void __user *from, unsigned long n)
{
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 3/9] x86/umip: Be stricter in fixup_umip_exception()
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
fixup_umip_exception() calls copy_to_user() and checks whether
the returned value is strictly positive.
A subsequent patch will change the return of copy_to_user() to
return -EFAULT in case of error.
Change the test to checking that the result is not 0.
At the time being copy_to_user() return an unsigned value so
'strictly positive' is the same as 'not 0'.
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
arch/x86/kernel/umip.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/umip.c b/arch/x86/kernel/umip.c
index 3ce99cbcf187..dfff28ea1dea 100644
--- a/arch/x86/kernel/umip.c
+++ b/arch/x86/kernel/umip.c
@@ -409,7 +409,7 @@ bool fixup_umip_exception(struct pt_regs *regs)
return false;
nr_copied = copy_to_user(uaddr, dummy_data, dummy_data_size);
- if (nr_copied > 0) {
+ if (nr_copied) {
/*
* If copy fails, send a signal and tell caller that
* fault was fixed up.
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 4/9] uaccess: Introduce copy_{to/from}_user_partial()
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
Today there are approximately 3000 calls for copy_to_user() and
3000 calls to copy_from_user().
The majority of callers of copy_{to/from}_user() don't care about the
return value, they only check whether it is 0 or not, and when it is
not 0 they handle it as a -EACCES.
In order to allow better optimisation of copy_{to/from}_user() when
the size of the copy is known at build time, create new fonctions
named copy_{to/from}_user_partial() to be used by the few callers
that are interested in partial copies and need to now how many
bytes remain at the end of the copy.
For the time being it is just the same as copy_{to/from}_user().
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
include/linux/uaccess.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index bd1201c81d94..2d37173782b3 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -221,6 +221,8 @@ copy_from_user(void *to, const void __user *from, unsigned long n)
return _inline_copy_from_user(to, from, n);
}
+#define copy_from_user_partial copy_from_user
+
static __always_inline unsigned long __must_check
copy_to_user(void __user *to, const void *from, unsigned long n)
{
@@ -233,6 +235,8 @@ copy_to_user(void __user *to, const void *from, unsigned long n)
return _inline_copy_to_user(to, from, n);
}
+#define copy_to_user_partial copy_to_user
+
#ifndef copy_mc_to_kernel
/*
* Without arch opt-in this generic copy_mc_to_kernel() will not handle
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 5/9] uaccess: Switch to copy_{to/from}_user_partial() when relevant
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
In a subsequent patch, copy_{to/from}_user() will be modified to
return -EFAULT when copy fails.
Among the 6000 calls to copy_{to/from}_user(), around 2% rely on
copy_{to/from}_user() doing partial copies and returning amount of not
copied bytes. Change those users to copy_{to/from}_user_partial().
This change was done based on whether callers assign the returned value
to a variable or just check whether the return value is 0 or not.
Several of them only use it for debug to print the amount of bytes not
copied. Those could maybe be changed to stop reporting that amount and
not be converted to partial copy.
Some not trivial handling might have been unecessarily converted. This
is not a problem and they can be converted back later for better
performance.
The callers where located with following commands then reviewed one by
one:
sed -i s/"return copy_to_user("/"return copy_to_user_partial("/g `git grep -l "return copy_to_user("`
sed -i s/" = copy_to_user("/" = copy_to_user_partial("/g `git grep -l " = copy_to_user("`
sed -i s/" += copy_to_user("/" += copy_to_user_partial("/g `git grep -l " += copy_to_user("`
sed -i s/" -= copy_to_user("/" -= copy_to_user_partial("/g `git grep -l " -= copy_to_user("`
Then the same was done with copy_from_user().
During the review, patterns like the following were rejected and kept
as is:
- return copy_to_user(osf_stat, &tmp, sizeof(tmp)) ? -EFAULT : 0;
+ return copy_to_user_partial(osf_stat, &tmp, sizeof(tmp)) ? -EFAULT : 0;
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
arch/alpha/kernel/osf_sys.c | 4 ++--
arch/alpha/kernel/termios.c | 2 +-
arch/arc/kernel/disasm.c | 2 +-
arch/arm64/include/asm/gcs.h | 2 +-
arch/arm64/kernel/signal32.c | 2 +-
arch/mips/kernel/rtlx.c | 8 ++++----
arch/mips/kernel/vpe.c | 2 +-
arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 ++--
arch/powerpc/kvm/book3s_64_mmu_radix.c | 4 ++--
arch/powerpc/kvm/book3s_hv.c | 2 +-
arch/riscv/kernel/signal.c | 2 +-
arch/s390/include/asm/idals.h | 8 ++++----
arch/sparc/kernel/termios.c | 2 +-
arch/um/kernel/process.c | 2 +-
arch/x86/lib/insn-eval.c | 2 +-
arch/x86/um/signal.c | 2 +-
drivers/android/binder_alloc.c | 2 +-
drivers/comedi/comedi_fops.c | 4 ++--
drivers/dma/idxd/cdev.c | 2 +-
drivers/firmware/efi/test/efi_test.c | 2 +-
drivers/fsi/fsi-scom.c | 2 +-
.../amd/display/amdgpu_dm/amdgpu_dm_debugfs.c | 2 +-
drivers/gpu/drm/i915/gt/intel_sseu.c | 4 ++--
drivers/gpu/drm/i915/i915_gem.c | 4 ++--
drivers/hwtracing/intel_th/msu.c | 2 +-
drivers/misc/ibmvmc.c | 2 +-
drivers/misc/vmw_vmci/vmci_host.c | 2 +-
drivers/most/most_cdev.c | 2 +-
drivers/net/ieee802154/ca8210.c | 4 ++--
drivers/net/wireless/ath/wil6210/debugfs.c | 2 +-
.../wireless/intel/iwlwifi/pcie/gen1_2/trans.c | 2 +-
drivers/net/wireless/ti/wlcore/debugfs.c | 2 +-
drivers/ps3/ps3-lpm.c | 2 +-
drivers/s390/crypto/zcrypt_api.h | 4 ++--
drivers/spi/spidev.c | 2 +-
.../staging/media/atomisp/pci/atomisp_cmd.c | 8 ++++----
drivers/tty/tty_ioctl.c | 14 +++++++-------
drivers/tty/vt/vc_screen.c | 4 ++--
drivers/usb/gadget/function/f_hid.c | 4 ++--
drivers/usb/gadget/function/f_printer.c | 2 +-
drivers/vfio/vfio_iommu_type1.c | 4 ++--
drivers/xen/xenbus/xenbus_dev_frontend.c | 2 +-
fs/namespace.c | 2 +-
fs/ocfs2/dlmfs/dlmfs.c | 2 +-
fs/proc/base.c | 4 ++--
include/linux/bpfptr.h | 2 +-
include/linux/sockptr.h | 4 ++--
ipc/msg.c | 8 ++++----
ipc/sem.c | 8 ++++----
ipc/shm.c | 18 +++++++++---------
kernel/regset.c | 2 +-
kernel/sys.c | 4 ++--
lib/kfifo.c | 8 ++++----
mm/kasan/kasan_test_c.c | 4 ++--
mm/memory.c | 2 +-
net/x25/af_x25.c | 2 +-
rust/helpers/uaccess.c | 4 ++--
sound/pci/emu10k1/emufx.c | 4 ++--
sound/pci/rme9652/hdsp.c | 6 +++---
sound/soc/intel/avs/probes.c | 6 +++---
sound/soc/sof/compress.c | 12 ++++++------
sound/soc/sof/sof-client-probes.c | 6 +++---
62 files changed, 122 insertions(+), 122 deletions(-)
diff --git a/arch/alpha/kernel/osf_sys.c b/arch/alpha/kernel/osf_sys.c
index 7b6543d2cca3..c8ea39fdbb9f 100644
--- a/arch/alpha/kernel/osf_sys.c
+++ b/arch/alpha/kernel/osf_sys.c
@@ -944,7 +944,7 @@ get_tv32(struct timespec64 *o, struct timeval32 __user *i)
static inline long
put_tv32(struct timeval32 __user *o, struct timespec64 *i)
{
- return copy_to_user(o, &(struct timeval32){
+ return copy_to_user_partial(o, &(struct timeval32){
.tv_sec = i->tv_sec,
.tv_usec = i->tv_nsec / NSEC_PER_USEC},
sizeof(struct timeval32));
@@ -953,7 +953,7 @@ put_tv32(struct timeval32 __user *o, struct timespec64 *i)
static inline long
put_tv_to_tv32(struct timeval32 __user *o, struct __kernel_old_timeval *i)
{
- return copy_to_user(o, &(struct timeval32){
+ return copy_to_user_partial(o, &(struct timeval32){
.tv_sec = i->tv_sec,
.tv_usec = i->tv_usec},
sizeof(struct timeval32));
diff --git a/arch/alpha/kernel/termios.c b/arch/alpha/kernel/termios.c
index a4c29a22edf7..a3693c29a0fd 100644
--- a/arch/alpha/kernel/termios.c
+++ b/arch/alpha/kernel/termios.c
@@ -52,5 +52,5 @@ int kernel_termios_to_user_termio(struct termio __user *termio,
v.c_cc[_VEOL2] = termios->c_cc[VEOL2];
v.c_cc[_VSWTC] = termios->c_cc[VSWTC];
- return copy_to_user(termio, &v, sizeof(struct termio));
+ return copy_to_user_partial(termio, &v, sizeof(struct termio));
}
diff --git a/arch/arc/kernel/disasm.c b/arch/arc/kernel/disasm.c
index ccc7e8c39eb3..a3ef9d079e7f 100644
--- a/arch/arc/kernel/disasm.c
+++ b/arch/arc/kernel/disasm.c
@@ -34,7 +34,7 @@ void __kprobes disasm_instr(unsigned long addr, struct disasm_state *state,
/* This fetches the upper part of the 32 bit instruction
* in both the cases of Little Endian or Big Endian configurations. */
if (userspace) {
- bytes_not_copied = copy_from_user(ins_buf,
+ bytes_not_copied = copy_from_user_partial(ins_buf,
(const void __user *) addr, 8);
if (bytes_not_copied > 6)
goto fault;
diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h
index 8fa0707069e8..7ee23a8130b0 100644
--- a/arch/arm64/include/asm/gcs.h
+++ b/arch/arm64/include/asm/gcs.h
@@ -139,7 +139,7 @@ static inline u64 get_user_gcs(unsigned long __user *addr, int *err)
/* Ensure previous GCS operation are visible before we read the page */
gcsb_dsync();
- ret = copy_from_user(&load, addr, sizeof(load));
+ ret = copy_from_user_partial(&load, addr, sizeof(load));
if (ret != 0)
*err = ret;
return load;
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index bb3b526ff43f..7016d2a3bb76 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -53,7 +53,7 @@ static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
cset.sig[0] = set->sig[0] & 0xffffffffull;
cset.sig[1] = set->sig[0] >> 32;
- return copy_to_user(uset, &cset, sizeof(*uset));
+ return copy_to_user_partial(uset, &cset, sizeof(*uset));
}
static inline int get_sigset_t(sigset_t *set,
diff --git a/arch/mips/kernel/rtlx.c b/arch/mips/kernel/rtlx.c
index 18c509c59f33..bc468064194d 100644
--- a/arch/mips/kernel/rtlx.c
+++ b/arch/mips/kernel/rtlx.c
@@ -262,13 +262,13 @@ ssize_t rtlx_read(int index, void __user *buff, size_t count)
/* then how much from the read pointer onwards */
fl = min(count, (size_t)lx->buffer_size - lx->lx_read);
- failed = copy_to_user(buff, lx->lx_buffer + lx->lx_read, fl);
+ failed = copy_to_user_partial(buff, lx->lx_buffer + lx->lx_read, fl);
if (failed)
goto out;
/* and if there is anything left at the beginning of the buffer */
if (count - fl)
- failed = copy_to_user(buff + fl, lx->lx_buffer, count - fl);
+ failed = copy_to_user_partial(buff + fl, lx->lx_buffer, count - fl);
out:
count -= failed;
@@ -304,13 +304,13 @@ ssize_t rtlx_write(int index, const void __user *buffer, size_t count)
/* first bit from write pointer to the end of the buffer, or count */
fl = min(count, (size_t) rt->buffer_size - rt->rt_write);
- failed = copy_from_user(rt->rt_buffer + rt->rt_write, buffer, fl);
+ failed = copy_from_user_partial(rt->rt_buffer + rt->rt_write, buffer, fl);
if (failed)
goto out;
/* if there's any left copy to the beginning of the buffer */
if (count - fl)
- failed = copy_from_user(rt->rt_buffer, buffer + fl, count - fl);
+ failed = copy_from_user_partial(rt->rt_buffer, buffer + fl, count - fl);
out:
count -= failed;
diff --git a/arch/mips/kernel/vpe.c b/arch/mips/kernel/vpe.c
index b05ee21a1d67..5a8d72d6c80c 100644
--- a/arch/mips/kernel/vpe.c
+++ b/arch/mips/kernel/vpe.c
@@ -854,7 +854,7 @@ static ssize_t vpe_write(struct file *file, const char __user *buffer,
return -ENOMEM;
}
- count -= copy_from_user(v->pbuffer + v->len, buffer, count);
+ count -= copy_from_user_partial(v->pbuffer + v->len, buffer, count);
if (!count)
return -EFAULT;
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index 2ccb3d138f46..1c43c7b8e801 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -2028,7 +2028,7 @@ static ssize_t debugfs_htab_read(struct file *file, char __user *buf,
n = p->chars_left;
if (n > len)
n = len;
- r = copy_to_user(buf, p->buf + p->buf_index, n);
+ r = copy_to_user_partial(buf, p->buf + p->buf_index, n);
n -= r;
p->chars_left -= n;
p->buf_index += n;
@@ -2068,7 +2068,7 @@ static ssize_t debugfs_htab_read(struct file *file, char __user *buf,
p->chars_left = n;
if (n > len)
n = len;
- r = copy_to_user(buf, p->buf, n);
+ r = copy_to_user_partial(buf, p->buf, n);
n -= r;
p->chars_left -= n;
p->buf_index = n;
diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c
index 933fc7cb9afc..0a27e018d27b 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c
@@ -1307,7 +1307,7 @@ static ssize_t debugfs_radix_read(struct file *file, char __user *buf,
n = p->chars_left;
if (n > len)
n = len;
- r = copy_to_user(buf, p->buf + p->buf_index, n);
+ r = copy_to_user_partial(buf, p->buf + p->buf_index, n);
n -= r;
p->chars_left -= n;
p->buf_index += n;
@@ -1407,7 +1407,7 @@ static ssize_t debugfs_radix_read(struct file *file, char __user *buf,
p->chars_left = n;
if (n > len)
n = len;
- r = copy_to_user(buf, p->buf, n);
+ r = copy_to_user_partial(buf, p->buf, n);
n -= r;
p->chars_left -= n;
p->buf_index = n;
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 61dbeea317f3..4c7a8f687c99 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2916,7 +2916,7 @@ static ssize_t debugfs_timings_read(struct file *file, char __user *buf,
return 0;
if (len > p->buflen - pos)
len = p->buflen - pos;
- n = copy_to_user(buf, p->buf + pos, len);
+ n = copy_to_user_partial(buf, p->buf + pos, len);
if (n) {
if (n == len)
return -EFAULT;
diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
index 59784dc117e4..4630dbad7428 100644
--- a/arch/riscv/kernel/signal.c
+++ b/arch/riscv/kernel/signal.c
@@ -139,7 +139,7 @@ static long __restore_v_state(struct pt_regs *regs, void __user *sc_vec)
* Copy the whole vector content from user space datap. Use
* copy_from_user to prevent information leak.
*/
- return copy_from_user(current->thread.vstate.datap, datap, riscv_v_vsize);
+ return copy_from_user_partial(current->thread.vstate.datap, datap, riscv_v_vsize);
}
static long save_cfiss_state(struct pt_regs *regs, void __user *sc_cfi)
diff --git a/arch/s390/include/asm/idals.h b/arch/s390/include/asm/idals.h
index 06e1ec2afd5a..d86f4eb1ce42 100644
--- a/arch/s390/include/asm/idals.h
+++ b/arch/s390/include/asm/idals.h
@@ -301,14 +301,14 @@ static inline size_t idal_buffer_to_user(struct idal_buffer *ib, void __user *to
BUG_ON(count > ib->size);
for (i = 0; count > IDA_BLOCK_SIZE; i++) {
vaddr = dma64_to_virt(ib->data[i]);
- left = copy_to_user(to, vaddr, IDA_BLOCK_SIZE);
+ left = copy_to_user_partial(to, vaddr, IDA_BLOCK_SIZE);
if (left)
return left + count - IDA_BLOCK_SIZE;
to = (void __user *)to + IDA_BLOCK_SIZE;
count -= IDA_BLOCK_SIZE;
}
vaddr = dma64_to_virt(ib->data[i]);
- return copy_to_user(to, vaddr, count);
+ return copy_to_user_partial(to, vaddr, count);
}
/*
@@ -323,14 +323,14 @@ static inline size_t idal_buffer_from_user(struct idal_buffer *ib, const void __
BUG_ON(count > ib->size);
for (i = 0; count > IDA_BLOCK_SIZE; i++) {
vaddr = dma64_to_virt(ib->data[i]);
- left = copy_from_user(vaddr, from, IDA_BLOCK_SIZE);
+ left = copy_from_user_partial(vaddr, from, IDA_BLOCK_SIZE);
if (left)
return left + count - IDA_BLOCK_SIZE;
from = (void __user *)from + IDA_BLOCK_SIZE;
count -= IDA_BLOCK_SIZE;
}
vaddr = dma64_to_virt(ib->data[i]);
- return copy_from_user(vaddr, from, count);
+ return copy_from_user_partial(vaddr, from, count);
}
#endif
diff --git a/arch/sparc/kernel/termios.c b/arch/sparc/kernel/termios.c
index ee64965c27cd..db9c07b7d5ee 100644
--- a/arch/sparc/kernel/termios.c
+++ b/arch/sparc/kernel/termios.c
@@ -27,7 +27,7 @@ int kernel_termios_to_user_termio(struct termio __user *termio,
v.c_cc[_VMIN] = termios->c_cc[VMIN];
v.c_cc[_VTIME] = termios->c_cc[VTIME];
}
- return copy_to_user(termio, &v, sizeof(struct termio));
+ return copy_to_user_partial(termio, &v, sizeof(struct termio));
}
int user_termios_to_kernel_termios(struct ktermios *k,
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 63b38a3f73f7..d41625dfa00b 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -252,7 +252,7 @@ EXPORT_SYMBOL(uml_strdup);
int copy_from_user_proc(void *to, void __user *from, int size)
{
- return copy_from_user(to, from, size);
+ return copy_from_user_partial(to, from, size);
}
int singlestepping(void)
diff --git a/arch/x86/lib/insn-eval.c b/arch/x86/lib/insn-eval.c
index e03eeec55cfe..e7cb03ab26f1 100644
--- a/arch/x86/lib/insn-eval.c
+++ b/arch/x86/lib/insn-eval.c
@@ -1512,7 +1512,7 @@ int insn_fetch_from_user(struct pt_regs *regs, unsigned char buf[MAX_INSN_SIZE])
if (insn_get_effective_ip(regs, &ip))
return -EINVAL;
- not_copied = copy_from_user(buf, (void __user *)ip, MAX_INSN_SIZE);
+ not_copied = copy_from_user_partial(buf, (void __user *)ip, MAX_INSN_SIZE);
return MAX_INSN_SIZE - not_copied;
}
diff --git a/arch/x86/um/signal.c b/arch/x86/um/signal.c
index 2934e170b0fe..e0fab7c1625b 100644
--- a/arch/x86/um/signal.c
+++ b/arch/x86/um/signal.c
@@ -40,7 +40,7 @@ static int copy_sc_from_user(struct pt_regs *regs,
/* Always make any pending restarted system calls return -EINTR */
current->restart_block.fn = do_no_restart_syscall;
- err = copy_from_user(&sc, from, sizeof(sc));
+ err = copy_from_user_partial(&sc, from, sizeof(sc));
if (err)
return err;
diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
index e4488ad86a65..8ba9c57b489c 100644
--- a/drivers/android/binder_alloc.c
+++ b/drivers/android/binder_alloc.c
@@ -1346,7 +1346,7 @@ binder_alloc_copy_user_to_buffer(struct binder_alloc *alloc,
buffer_offset, &pgoff);
size = min_t(size_t, bytes, PAGE_SIZE - pgoff);
kptr = kmap_local_page(page) + pgoff;
- ret = copy_from_user(kptr, from, size);
+ ret = copy_from_user_partial(kptr, from, size);
kunmap_local(kptr);
if (ret)
return bytes - size + ret;
diff --git a/drivers/comedi/comedi_fops.c b/drivers/comedi/comedi_fops.c
index c09bbe04be6c..272fdc54fb81 100644
--- a/drivers/comedi/comedi_fops.c
+++ b/drivers/comedi/comedi_fops.c
@@ -2659,7 +2659,7 @@ static unsigned int comedi_buf_copy_to_user(struct comedi_subdevice *s,
unsigned int copy_amount = min(n, PAGE_SIZE - offset);
unsigned int uncopied;
- uncopied = copy_to_user(dest, buf_page_list[page].virt_addr +
+ uncopied = copy_to_user_partial(dest, buf_page_list[page].virt_addr +
offset, copy_amount);
copy_amount -= uncopied;
n -= copy_amount;
@@ -2687,7 +2687,7 @@ static unsigned int comedi_buf_copy_from_user(struct comedi_subdevice *s,
unsigned int copy_amount = min(n, PAGE_SIZE - offset);
unsigned int uncopied;
- uncopied = copy_from_user(buf_page_list[page].virt_addr +
+ uncopied = copy_from_user_partial(buf_page_list[page].virt_addr +
offset, src, copy_amount);
copy_amount -= uncopied;
n -= copy_amount;
diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c
index 0366c7cf3502..ac79bab6d6c3 100644
--- a/drivers/dma/idxd/cdev.c
+++ b/drivers/dma/idxd/cdev.c
@@ -751,7 +751,7 @@ int idxd_copy_cr(struct idxd_wq *wq, ioasid_t pasid, unsigned long addr,
* to addr in the mm.
*/
kthread_use_mm(mm);
- left = copy_to_user((void __user *)addr + status_size, cr + status_size,
+ left = copy_to_user_partial((void __user *)addr + status_size, cr + status_size,
len - status_size);
/*
* Copy status only after the rest of completion record is copied
diff --git a/drivers/firmware/efi/test/efi_test.c b/drivers/firmware/efi/test/efi_test.c
index d54d6a671326..43b280ceb955 100644
--- a/drivers/firmware/efi/test/efi_test.c
+++ b/drivers/firmware/efi/test/efi_test.c
@@ -133,7 +133,7 @@ copy_ucs2_to_user_len(efi_char16_t __user *dst, efi_char16_t *src, size_t len)
if (!src)
return 0;
- return copy_to_user(dst, src, len);
+ return copy_to_user_partial(dst, src, len);
}
static long efi_runtime_get_variable(unsigned long arg)
diff --git a/drivers/fsi/fsi-scom.c b/drivers/fsi/fsi-scom.c
index bb4d3700c934..370ec75b20e6 100644
--- a/drivers/fsi/fsi-scom.c
+++ b/drivers/fsi/fsi-scom.c
@@ -332,7 +332,7 @@ static ssize_t scom_read(struct file *filep, char __user *buf, size_t len,
return rc;
}
- rc = copy_to_user(buf, &val, len);
+ rc = copy_to_user_partial(buf, &val, len);
if (rc)
dev_dbg(dev, "copy to user failed:%d\n", rc);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
index 2409ac72b166..712605ec7ecc 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
@@ -1346,7 +1346,7 @@ static ssize_t dp_sdp_message_debugfs_write(struct file *f, const char __user *b
acrtc_state = to_dm_crtc_state(connector->base.state->crtc->state);
- r = copy_from_user(data, buf, write_size);
+ r = copy_from_user_partial(data, buf, write_size);
write_size -= r;
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c
index 656a499b2706..3f5b450a914a 100644
--- a/drivers/gpu/drm/i915/gt/intel_sseu.c
+++ b/drivers/gpu/drm/i915/gt/intel_sseu.c
@@ -114,7 +114,7 @@ int intel_sseu_copy_eumask_to_user(void __user *to,
}
}
- return copy_to_user(to, eu_mask, len);
+ return copy_to_user_partial(to, eu_mask, len);
}
/**
@@ -146,7 +146,7 @@ int intel_sseu_copy_ssmask_to_user(void __user *to,
}
}
- return copy_to_user(to, ss_mask, len);
+ return copy_to_user_partial(to, ss_mask, len);
}
static void gen11_compute_sseu_info(struct sseu_dev_info *sseu,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a432daf8038a..c1c2e762498f 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -291,7 +291,7 @@ gtt_user_read(struct io_mapping *mapping,
io_mapping_unmap_atomic(vaddr);
if (unwritten) {
vaddr = io_mapping_map_wc(mapping, base, PAGE_SIZE);
- unwritten = copy_to_user(user_data,
+ unwritten = copy_to_user_partial(user_data,
(void __force *)vaddr + offset,
length);
io_mapping_unmap(vaddr);
@@ -525,7 +525,7 @@ ggtt_write(struct io_mapping *mapping,
io_mapping_unmap_atomic(vaddr);
if (unwritten) {
vaddr = io_mapping_map_wc(mapping, base, PAGE_SIZE);
- unwritten = copy_from_user((void __force *)vaddr + offset,
+ unwritten = copy_from_user_partial((void __force *)vaddr + offset,
user_data, length);
io_mapping_unmap(vaddr);
}
diff --git a/drivers/hwtracing/intel_th/msu.c b/drivers/hwtracing/intel_th/msu.c
index a82cf74f39ad..9b97b71b44f1 100644
--- a/drivers/hwtracing/intel_th/msu.c
+++ b/drivers/hwtracing/intel_th/msu.c
@@ -1457,7 +1457,7 @@ static unsigned long msc_win_to_user(void *data, void *src, size_t len)
struct msc_win_to_user_struct *u = data;
unsigned long ret;
- ret = copy_to_user(u->buf + u->offset, src, len);
+ ret = copy_to_user_partial(u->buf + u->offset, src, len);
u->offset += len - ret;
return ret;
diff --git a/drivers/misc/ibmvmc.c b/drivers/misc/ibmvmc.c
index beb18c34f20d..e1d99354dd29 100644
--- a/drivers/misc/ibmvmc.c
+++ b/drivers/misc/ibmvmc.c
@@ -1112,7 +1112,7 @@ static ssize_t ibmvmc_write(struct file *file, const char *buffer,
while (c > 0) {
bytes = min_t(size_t, c, vmc_buffer->size);
- bytes -= copy_from_user(buf, p, bytes);
+ bytes -= copy_from_user_partial(buf, p, bytes);
if (!bytes) {
ret = -EFAULT;
goto out;
diff --git a/drivers/misc/vmw_vmci/vmci_host.c b/drivers/misc/vmw_vmci/vmci_host.c
index b71ca1bf0a20..bd502edbc173 100644
--- a/drivers/misc/vmw_vmci/vmci_host.c
+++ b/drivers/misc/vmw_vmci/vmci_host.c
@@ -213,7 +213,7 @@ static int drv_cp_harray_to_user(void __user *user_buf_uva,
*user_buf_size = array_size * sizeof(*handles);
if (*user_buf_size)
- *retval = copy_to_user(user_buf_uva,
+ *retval = copy_to_user_partial(user_buf_uva,
vmci_handle_arr_get_handles
(handle_array), *user_buf_size);
diff --git a/drivers/most/most_cdev.c b/drivers/most/most_cdev.c
index 5df508d8d60a..969c865ccbef 100644
--- a/drivers/most/most_cdev.c
+++ b/drivers/most/most_cdev.c
@@ -265,7 +265,7 @@ comp_read(struct file *filp, char __user *buf, size_t count, loff_t *offset)
count,
mbo->processed_length - c->mbo_offs);
- not_copied = copy_to_user(buf,
+ not_copied = copy_to_user_partial(buf,
mbo->virt_address + c->mbo_offs,
to_copy);
diff --git a/drivers/net/ieee802154/ca8210.c b/drivers/net/ieee802154/ca8210.c
index ed4178155a5d..d474a008c73e 100644
--- a/drivers/net/ieee802154/ca8210.c
+++ b/drivers/net/ieee802154/ca8210.c
@@ -2460,7 +2460,7 @@ static ssize_t ca8210_test_int_user_write(
return -EBADE;
}
- ret = copy_from_user(command, in_buf, len);
+ ret = copy_from_user_partial(command, in_buf, len);
if (ret) {
dev_err(
&priv->spi->dev,
@@ -2548,7 +2548,7 @@ static ssize_t ca8210_test_int_user_read(
cmdlen = fifo_buffer[1];
bytes_not_copied = cmdlen + 2;
- bytes_not_copied = copy_to_user(buf, fifo_buffer, bytes_not_copied);
+ bytes_not_copied = copy_to_user_partial(buf, fifo_buffer, bytes_not_copied);
if (bytes_not_copied > 0) {
dev_err(
&priv->spi->dev,
diff --git a/drivers/net/wireless/ath/wil6210/debugfs.c b/drivers/net/wireless/ath/wil6210/debugfs.c
index b8cb736a7185..f2130248fb7f 100644
--- a/drivers/net/wireless/ath/wil6210/debugfs.c
+++ b/drivers/net/wireless/ath/wil6210/debugfs.c
@@ -659,7 +659,7 @@ static ssize_t wil_read_file_ioblob(struct file *file, char __user *user_buf,
wil_memcpy_fromio_32(buf, (const void __iomem *)
wil_blob->blob.data + aligned_pos, aligned_count);
- ret = copy_to_user(user_buf, buf + unaligned_bytes, count);
+ ret = copy_to_user_partial(user_buf, buf + unaligned_bytes, count);
wil_mem_access_unlock(wil);
wil_pm_runtime_put(wil);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/gen1_2/trans.c b/drivers/net/wireless/intel/iwlwifi/pcie/gen1_2/trans.c
index a05f60f9224b..66ddaa0d8e36 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/gen1_2/trans.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/gen1_2/trans.c
@@ -3060,7 +3060,7 @@ static bool iwl_write_to_user_buf(char __user *user_buf, ssize_t count,
if (*size > buf_size_left)
*size = buf_size_left;
- *size -= copy_to_user(user_buf, buf, *size);
+ *size -= copy_to_user_partial(user_buf, buf, *size);
*bytes_copied += *size;
if (buf_size_left == *size)
diff --git a/drivers/net/wireless/ti/wlcore/debugfs.c b/drivers/net/wireless/ti/wlcore/debugfs.c
index bbfd2725215b..d359baea5100 100644
--- a/drivers/net/wireless/ti/wlcore/debugfs.c
+++ b/drivers/net/wireless/ti/wlcore/debugfs.c
@@ -1088,7 +1088,7 @@ static ssize_t dev_mem_read(struct file *file,
mutex_unlock(&wl->mutex);
if (ret == 0) {
- ret = copy_to_user(user_buf, buf, bytes);
+ ret = copy_to_user_partial(user_buf, buf, bytes);
if (ret < bytes) {
bytes -= ret;
*ppos += bytes;
diff --git a/drivers/ps3/ps3-lpm.c b/drivers/ps3/ps3-lpm.c
index f8d8f607134a..5a2b150cda49 100644
--- a/drivers/ps3/ps3-lpm.c
+++ b/drivers/ps3/ps3-lpm.c
@@ -999,7 +999,7 @@ int ps3_lpm_copy_tb_to_user(unsigned long offset, void __user *buf,
return result == LV1_WRONG_STATE ? -EBUSY : -EINVAL;
}
- result = copy_to_user(buf, lpm_priv->tb_cache, tmp);
+ result = copy_to_user_partial(buf, lpm_priv->tb_cache, tmp);
if (result) {
dev_dbg(sbd_core(), "%s:%u: 0x%llx bytes at 0x%p\n",
diff --git a/drivers/s390/crypto/zcrypt_api.h b/drivers/s390/crypto/zcrypt_api.h
index 6ef8850a42df..61a5de90c354 100644
--- a/drivers/s390/crypto/zcrypt_api.h
+++ b/drivers/s390/crypto/zcrypt_api.h
@@ -185,7 +185,7 @@ static inline unsigned long z_copy_from_user(bool userspace,
unsigned long n)
{
if (likely(userspace))
- return copy_from_user(to, from, n);
+ return copy_from_user_partial(to, from, n);
memcpy(to, (void __force *)from, n);
return 0;
}
@@ -196,7 +196,7 @@ static inline unsigned long z_copy_to_user(bool userspace,
unsigned long n)
{
if (likely(userspace))
- return copy_to_user(to, from, n);
+ return copy_to_user_partial(to, from, n);
memcpy((void __force *)to, from, n);
return 0;
}
diff --git a/drivers/spi/spidev.c b/drivers/spi/spidev.c
index 638221178384..5b42fabcf4c4 100644
--- a/drivers/spi/spidev.c
+++ b/drivers/spi/spidev.c
@@ -157,7 +157,7 @@ spidev_read(struct file *filp, char __user *buf, size_t count, loff_t *f_pos)
if (status > 0) {
unsigned long missing;
- missing = copy_to_user(buf, spidev->rx_buffer, status);
+ missing = copy_to_user_partial(buf, spidev->rx_buffer, status);
if (missing == status)
status = -EFAULT;
else
diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.c b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
index fec369575d88..10a7aff375a9 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_cmd.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
@@ -1491,7 +1491,7 @@ int atomisp_gdc_cac_table(struct atomisp_sub_device *asd, int flag,
}
for (i = 0; i < IA_CSS_MORPH_TABLE_NUM_PLANES; i++) {
- ret = copy_from_user(tab->coordinates_x[i],
+ ret = copy_from_user_partial(tab->coordinates_x[i],
config->coordinates_x[i],
config->height * config->width *
sizeof(*config->coordinates_x[i]));
@@ -1502,7 +1502,7 @@ int atomisp_gdc_cac_table(struct atomisp_sub_device *asd, int flag,
atomisp_css_morph_table_free(tab);
return -EFAULT;
}
- ret = copy_from_user(tab->coordinates_y[i],
+ ret = copy_from_user_partial(tab->coordinates_y[i],
config->coordinates_y[i],
config->height * config->width *
sizeof(*config->coordinates_y[i]));
@@ -1709,7 +1709,7 @@ int atomisp_3a_stat(struct atomisp_sub_device *asd, int flag,
config->exp_id = s3a_buf->s3a_data->exp_id;
config->isp_config_id = s3a_buf->s3a_data->isp_config_id;
- ret = copy_to_user(config->data, asd->params.s3a_user_stat->data,
+ ret = copy_to_user_partial(config->data, asd->params.s3a_user_stat->data,
asd->params.s3a_output_bytes);
if (ret) {
dev_err(isp->dev, "copy to user failed: copied %lu bytes\n",
@@ -2031,7 +2031,7 @@ static unsigned int long copy_from_compatible(void *to, const void *from,
unsigned long n, bool from_user)
{
if (from_user)
- return copy_from_user(to, (void __user *)from, n);
+ return copy_from_user_partial(to, (void __user *)from, n);
else
memcpy(to, from, n);
return 0;
diff --git a/drivers/tty/tty_ioctl.c b/drivers/tty/tty_ioctl.c
index 90c70d8d14e3..cdc274c0ff81 100644
--- a/drivers/tty/tty_ioctl.c
+++ b/drivers/tty/tty_ioctl.c
@@ -388,29 +388,29 @@ __weak int kernel_termios_to_user_termio(struct termio __user *termio,
v.c_lflag = termios->c_lflag;
v.c_line = termios->c_line;
memcpy(v.c_cc, termios->c_cc, NCC);
- return copy_to_user(termio, &v, sizeof(struct termio));
+ return copy_to_user_partial(termio, &v, sizeof(struct termio));
}
#ifdef TCGETS2
__weak int user_termios_to_kernel_termios(struct ktermios *k,
struct termios2 __user *u)
{
- return copy_from_user(k, u, sizeof(struct termios2));
+ return copy_from_user_partial(k, u, sizeof(struct termios2));
}
__weak int kernel_termios_to_user_termios(struct termios2 __user *u,
struct ktermios *k)
{
- return copy_to_user(u, k, sizeof(struct termios2));
+ return copy_to_user_partial(u, k, sizeof(struct termios2));
}
__weak int user_termios_to_kernel_termios_1(struct ktermios *k,
struct termios __user *u)
{
- return copy_from_user(k, u, sizeof(struct termios));
+ return copy_from_user_partial(k, u, sizeof(struct termios));
}
__weak int kernel_termios_to_user_termios_1(struct termios __user *u,
struct ktermios *k)
{
- return copy_to_user(u, k, sizeof(struct termios));
+ return copy_to_user_partial(u, k, sizeof(struct termios));
}
#else
@@ -418,12 +418,12 @@ __weak int kernel_termios_to_user_termios_1(struct termios __user *u,
__weak int user_termios_to_kernel_termios(struct ktermios *k,
struct termios __user *u)
{
- return copy_from_user(k, u, sizeof(struct termios));
+ return copy_from_user_partial(k, u, sizeof(struct termios));
}
__weak int kernel_termios_to_user_termios(struct termios __user *u,
struct ktermios *k)
{
- return copy_to_user(u, k, sizeof(struct termios));
+ return copy_to_user_partial(u, k, sizeof(struct termios));
}
#endif /* TCGETS2 */
diff --git a/drivers/tty/vt/vc_screen.c b/drivers/tty/vt/vc_screen.c
index 4d2d46c95fef..e54c708149c3 100644
--- a/drivers/tty/vt/vc_screen.c
+++ b/drivers/tty/vt/vc_screen.c
@@ -450,7 +450,7 @@ vcs_read(struct file *file, char __user *buf, size_t count, loff_t *ppos)
*/
console_unlock();
- ret = copy_to_user(buf, con_buf + skip, this_round);
+ ret = copy_to_user_partial(buf, con_buf + skip, this_round);
console_lock();
if (ret) {
@@ -630,7 +630,7 @@ vcs_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos)
* in the write data from userspace safely.
*/
console_unlock();
- ret = copy_from_user(con_buf, buf, this_round);
+ ret = copy_from_user_partial(con_buf, buf, this_round);
console_lock();
if (ret) {
diff --git a/drivers/usb/gadget/function/f_hid.c b/drivers/usb/gadget/function/f_hid.c
index c5a12a6760ea..f22dd3697a46 100644
--- a/drivers/usb/gadget/function/f_hid.c
+++ b/drivers/usb/gadget/function/f_hid.c
@@ -349,7 +349,7 @@ static ssize_t f_hidg_intout_read(struct file *file, char __user *buffer,
spin_unlock_irqrestore(&hidg->read_spinlock, flags);
/* copy to user outside spinlock */
- count -= copy_to_user(buffer, req->buf + list->pos, count);
+ count -= copy_to_user_partial(buffer, req->buf + list->pos, count);
list->pos += count;
/*
@@ -410,7 +410,7 @@ static ssize_t f_hidg_ssreport_read(struct file *file, char __user *buffer,
spin_unlock_irqrestore(&hidg->read_spinlock, flags);
if (tmp_buf != NULL) {
- count -= copy_to_user(buffer, tmp_buf, count);
+ count -= copy_to_user_partial(buffer, tmp_buf, count);
kfree(tmp_buf);
} else {
count = -ENOMEM;
diff --git a/drivers/usb/gadget/function/f_printer.c b/drivers/usb/gadget/function/f_printer.c
index e4f7828ae75d..4fbed987b639 100644
--- a/drivers/usb/gadget/function/f_printer.c
+++ b/drivers/usb/gadget/function/f_printer.c
@@ -525,7 +525,7 @@ printer_read(struct file *fd, char __user *buf, size_t len, loff_t *ptr)
else
size = len;
- size -= copy_to_user(buf, current_rx_buf, size);
+ size -= copy_to_user_partial(buf, current_rx_buf, size);
bytes_copied += size;
len -= size;
buf += size;
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index c8151ba54de3..ad74a891aa80 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -3173,7 +3173,7 @@ static int vfio_iommu_type1_dma_rw_chunk(struct vfio_iommu *iommu,
vaddr = dma->vaddr + offset;
if (write) {
- *copied = copy_to_user((void __user *)vaddr, data,
+ *copied = copy_to_user_partial((void __user *)vaddr, data,
count) ? 0 : count;
if (*copied && iommu->dirty_page_tracking) {
unsigned long pgshift = __ffs(iommu->pgsize_bitmap);
@@ -3186,7 +3186,7 @@ static int vfio_iommu_type1_dma_rw_chunk(struct vfio_iommu *iommu,
(offset >> pgshift) + 1);
}
} else
- *copied = copy_from_user(data, (void __user *)vaddr,
+ *copied = copy_from_user_partial(data, (void __user *)vaddr,
count) ? 0 : count;
if (kthread)
kthread_unuse_mm(mm);
diff --git a/drivers/xen/xenbus/xenbus_dev_frontend.c b/drivers/xen/xenbus/xenbus_dev_frontend.c
index 61db6932a9d2..b1db90dac1d1 100644
--- a/drivers/xen/xenbus/xenbus_dev_frontend.c
+++ b/drivers/xen/xenbus/xenbus_dev_frontend.c
@@ -150,7 +150,7 @@ static ssize_t xenbus_file_read(struct file *filp,
while (i < len) {
size_t sz = min_t(size_t, len - i, rb->len - rb->cons);
- ret = copy_to_user(ubuf + i, &rb->msg[rb->cons], sz);
+ ret = copy_to_user_partial(ubuf + i, &rb->msg[rb->cons], sz);
i += sz - ret;
rb->cons += sz - ret;
diff --git a/fs/namespace.c b/fs/namespace.c
index fe919abd2f01..27afb73fef20 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -4033,7 +4033,7 @@ static void *copy_mount_options(const void __user * data)
if (!copy)
return ERR_PTR(-ENOMEM);
- left = copy_from_user(copy, data, PAGE_SIZE);
+ left = copy_from_user_partial(copy, data, PAGE_SIZE);
/*
* Not all architectures have an exact copy_from_user(). Resort to
diff --git a/fs/ocfs2/dlmfs/dlmfs.c b/fs/ocfs2/dlmfs/dlmfs.c
index 5821e33df78f..97c0b391b98e 100644
--- a/fs/ocfs2/dlmfs/dlmfs.c
+++ b/fs/ocfs2/dlmfs/dlmfs.c
@@ -255,7 +255,7 @@ static ssize_t dlmfs_file_write(struct file *filp,
if (!count)
return 0;
- bytes_left = copy_from_user(lvb_buf, buf, count);
+ bytes_left = copy_from_user_partial(lvb_buf, buf, count);
count -= bytes_left;
if (count)
user_dlm_write_lvb(inode, lvb_buf, count);
diff --git a/fs/proc/base.c b/fs/proc/base.c
index d9acfa89c894..49577662ae70 100644
--- a/fs/proc/base.c
+++ b/fs/proc/base.c
@@ -278,7 +278,7 @@ static ssize_t get_mm_proctitle(struct mm_struct *mm, char __user *buf,
len -= pos;
if (len > count)
len = count;
- len -= copy_to_user(buf, page+pos, len);
+ len -= copy_to_user_partial(buf, page+pos, len);
if (!len)
len = -EFAULT;
ret = len;
@@ -359,7 +359,7 @@ static ssize_t get_mm_cmdline(struct mm_struct *mm, char __user *buf,
got = access_remote_vm(mm, pos, page, size, FOLL_ANON);
if (got <= 0)
break;
- got -= copy_to_user(buf, page, got);
+ got -= copy_to_user_partial(buf, page, got);
if (unlikely(!got)) {
if (!len)
len = -EFAULT;
diff --git a/include/linux/bpfptr.h b/include/linux/bpfptr.h
index f6e0795db484..e4444d0f0cfe 100644
--- a/include/linux/bpfptr.h
+++ b/include/linux/bpfptr.h
@@ -50,7 +50,7 @@ static inline int copy_from_bpfptr_offset(void *dst, bpfptr_t src,
size_t offset, size_t size)
{
if (!bpfptr_is_kernel(src))
- return copy_from_user(dst, src.user + offset, size);
+ return copy_from_user_partial(dst, src.user + offset, size);
return copy_from_kernel_nofault(dst, src.kernel + offset, size);
}
diff --git a/include/linux/sockptr.h b/include/linux/sockptr.h
index 3e6c8e9d67ae..52ddddfe728d 100644
--- a/include/linux/sockptr.h
+++ b/include/linux/sockptr.h
@@ -45,7 +45,7 @@ static inline int copy_from_sockptr_offset(void *dst, sockptr_t src,
size_t offset, size_t size)
{
if (!sockptr_is_kernel(src))
- return copy_from_user(dst, src.user + offset, size);
+ return copy_from_user_partial(dst, src.user + offset, size);
memcpy(dst, src.kernel + offset, size);
return 0;
}
@@ -111,7 +111,7 @@ static inline int copy_to_sockptr_offset(sockptr_t dst, size_t offset,
const void *src, size_t size)
{
if (!sockptr_is_kernel(dst))
- return copy_to_user(dst.user + offset, src, size);
+ return copy_to_user_partial(dst.user + offset, src, size);
memcpy(dst.kernel + offset, src, size);
return 0;
}
diff --git a/ipc/msg.c b/ipc/msg.c
index 62996b97f0ac..39848238219d 100644
--- a/ipc/msg.c
+++ b/ipc/msg.c
@@ -322,7 +322,7 @@ copy_msqid_to_user(void __user *buf, struct msqid64_ds *in, int version)
{
switch (version) {
case IPC_64:
- return copy_to_user(buf, in, sizeof(*in));
+ return copy_to_user_partial(buf, in, sizeof(*in));
case IPC_OLD:
{
struct msqid_ds out;
@@ -355,7 +355,7 @@ copy_msqid_to_user(void __user *buf, struct msqid64_ds *in, int version)
out.msg_lspid = in->msg_lspid;
out.msg_lrpid = in->msg_lrpid;
- return copy_to_user(buf, &out, sizeof(out));
+ return copy_to_user_partial(buf, &out, sizeof(out));
}
default:
return -EINVAL;
@@ -712,7 +712,7 @@ static int copy_compat_msqid_to_user(void __user *buf, struct msqid64_ds *in,
v.msg_qbytes = in->msg_qbytes;
v.msg_lspid = in->msg_lspid;
v.msg_lrpid = in->msg_lrpid;
- return copy_to_user(buf, &v, sizeof(v));
+ return copy_to_user_partial(buf, &v, sizeof(v));
} else {
struct compat_msqid_ds v;
memset(&v, 0, sizeof(v));
@@ -725,7 +725,7 @@ static int copy_compat_msqid_to_user(void __user *buf, struct msqid64_ds *in,
v.msg_qbytes = in->msg_qbytes;
v.msg_lspid = in->msg_lspid;
v.msg_lrpid = in->msg_lrpid;
- return copy_to_user(buf, &v, sizeof(v));
+ return copy_to_user_partial(buf, &v, sizeof(v));
}
}
diff --git a/ipc/sem.c b/ipc/sem.c
index 6cdf862b1f5c..3b56086ba07d 100644
--- a/ipc/sem.c
+++ b/ipc/sem.c
@@ -1196,7 +1196,7 @@ static unsigned long copy_semid_to_user(void __user *buf, struct semid64_ds *in,
{
switch (version) {
case IPC_64:
- return copy_to_user(buf, in, sizeof(*in));
+ return copy_to_user_partial(buf, in, sizeof(*in));
case IPC_OLD:
{
struct semid_ds out;
@@ -1209,7 +1209,7 @@ static unsigned long copy_semid_to_user(void __user *buf, struct semid64_ds *in,
out.sem_ctime = in->sem_ctime;
out.sem_nsems = in->sem_nsems;
- return copy_to_user(buf, &out, sizeof(out));
+ return copy_to_user_partial(buf, &out, sizeof(out));
}
default:
return -EINVAL;
@@ -1759,7 +1759,7 @@ static int copy_compat_semid_to_user(void __user *buf, struct semid64_ds *in,
v.sem_ctime = lower_32_bits(in->sem_ctime);
v.sem_ctime_high = upper_32_bits(in->sem_ctime);
v.sem_nsems = in->sem_nsems;
- return copy_to_user(buf, &v, sizeof(v));
+ return copy_to_user_partial(buf, &v, sizeof(v));
} else {
struct compat_semid_ds v;
memset(&v, 0, sizeof(v));
@@ -1767,7 +1767,7 @@ static int copy_compat_semid_to_user(void __user *buf, struct semid64_ds *in,
v.sem_otime = in->sem_otime;
v.sem_ctime = in->sem_ctime;
v.sem_nsems = in->sem_nsems;
- return copy_to_user(buf, &v, sizeof(v));
+ return copy_to_user_partial(buf, &v, sizeof(v));
}
}
diff --git a/ipc/shm.c b/ipc/shm.c
index a95dae447707..1eb53c3df3b9 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -853,7 +853,7 @@ static inline unsigned long copy_shmid_to_user(void __user *buf, struct shmid64_
{
switch (version) {
case IPC_64:
- return copy_to_user(buf, in, sizeof(*in));
+ return copy_to_user_partial(buf, in, sizeof(*in));
case IPC_OLD:
{
struct shmid_ds out;
@@ -868,7 +868,7 @@ static inline unsigned long copy_shmid_to_user(void __user *buf, struct shmid64_
out.shm_lpid = in->shm_lpid;
out.shm_nattch = in->shm_nattch;
- return copy_to_user(buf, &out, sizeof(out));
+ return copy_to_user_partial(buf, &out, sizeof(out));
}
default:
return -EINVAL;
@@ -905,7 +905,7 @@ static inline unsigned long copy_shminfo_to_user(void __user *buf, struct shminf
{
switch (version) {
case IPC_64:
- return copy_to_user(buf, in, sizeof(*in));
+ return copy_to_user_partial(buf, in, sizeof(*in));
case IPC_OLD:
{
struct shminfo out;
@@ -920,7 +920,7 @@ static inline unsigned long copy_shminfo_to_user(void __user *buf, struct shminf
out.shmseg = in->shmseg;
out.shmall = in->shmall;
- return copy_to_user(buf, &out, sizeof(out));
+ return copy_to_user_partial(buf, &out, sizeof(out));
}
default:
return -EINVAL;
@@ -1359,7 +1359,7 @@ static int copy_compat_shminfo_to_user(void __user *buf, struct shminfo64 *in,
info.shmmni = in->shmmni;
info.shmseg = in->shmseg;
info.shmall = in->shmall;
- return copy_to_user(buf, &info, sizeof(info));
+ return copy_to_user_partial(buf, &info, sizeof(info));
} else {
struct shminfo info;
memset(&info, 0, sizeof(info));
@@ -1368,7 +1368,7 @@ static int copy_compat_shminfo_to_user(void __user *buf, struct shminfo64 *in,
info.shmmni = in->shmmni;
info.shmseg = in->shmseg;
info.shmall = in->shmall;
- return copy_to_user(buf, &info, sizeof(info));
+ return copy_to_user_partial(buf, &info, sizeof(info));
}
}
@@ -1384,7 +1384,7 @@ static int put_compat_shm_info(struct shm_info *ip,
info.shm_swp = ip->shm_swp;
info.swap_attempts = ip->swap_attempts;
info.swap_successes = ip->swap_successes;
- return copy_to_user(uip, &info, sizeof(info));
+ return copy_to_user_partial(uip, &info, sizeof(info));
}
static int copy_compat_shmid_to_user(void __user *buf, struct shmid64_ds *in,
@@ -1404,7 +1404,7 @@ static int copy_compat_shmid_to_user(void __user *buf, struct shmid64_ds *in,
v.shm_nattch = in->shm_nattch;
v.shm_cpid = in->shm_cpid;
v.shm_lpid = in->shm_lpid;
- return copy_to_user(buf, &v, sizeof(v));
+ return copy_to_user_partial(buf, &v, sizeof(v));
} else {
struct compat_shmid_ds v;
memset(&v, 0, sizeof(v));
@@ -1417,7 +1417,7 @@ static int copy_compat_shmid_to_user(void __user *buf, struct shmid64_ds *in,
v.shm_nattch = in->shm_nattch;
v.shm_cpid = in->shm_cpid;
v.shm_lpid = in->shm_lpid;
- return copy_to_user(buf, &v, sizeof(v));
+ return copy_to_user_partial(buf, &v, sizeof(v));
}
}
diff --git a/kernel/regset.c b/kernel/regset.c
index b2871fa68b2a..29c6d19c3465 100644
--- a/kernel/regset.c
+++ b/kernel/regset.c
@@ -70,7 +70,7 @@ int copy_regset_to_user(struct task_struct *target,
ret = regset_get_alloc(target, regset, size, &buf);
if (ret > 0)
- ret = copy_to_user(data, buf, ret) ? -EFAULT : 0;
+ ret = copy_to_user_partial(data, buf, ret) ? -EFAULT : 0;
kvfree(buf);
return ret;
}
diff --git a/kernel/sys.c b/kernel/sys.c
index 62e842055cc9..8e1ce8c26884 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -1343,7 +1343,7 @@ static int override_release(char __user *release, size_t len)
v = LINUX_VERSION_PATCHLEVEL + 60;
copy = clamp_t(size_t, len, 1, sizeof(buf));
copy = scnprintf(buf, copy, "2.6.%u%s", v, rest);
- ret = copy_to_user(release, buf, copy + 1);
+ ret = copy_to_user_partial(release, buf, copy + 1);
}
return ret;
}
@@ -1567,7 +1567,7 @@ SYSCALL_DEFINE2(getrlimit, unsigned int, resource, struct rlimit __user *, rlim)
ret = do_prlimit(current, resource, NULL, &value);
if (!ret)
- ret = copy_to_user(rlim, &value, sizeof(*rlim)) ? -EFAULT : 0;
+ ret = copy_to_user_partial(rlim, &value, sizeof(*rlim)) ? -EFAULT : 0;
return ret;
}
diff --git a/lib/kfifo.c b/lib/kfifo.c
index 2633f9cc336c..00c19a321aae 100644
--- a/lib/kfifo.c
+++ b/lib/kfifo.c
@@ -203,11 +203,11 @@ static unsigned long kfifo_copy_from_user(struct __kfifo *fifo,
}
l = min(len, size - off);
- ret = copy_from_user(fifo->data + off, from, l);
+ ret = copy_from_user_partial(fifo->data + off, from, l);
if (unlikely(ret))
ret = DIV_ROUND_UP(ret + len - l, esize);
else {
- ret = copy_from_user(fifo->data, from + l, len - l);
+ ret = copy_from_user_partial(fifo->data, from + l, len - l);
if (unlikely(ret))
ret = DIV_ROUND_UP(ret, esize);
}
@@ -263,11 +263,11 @@ static unsigned long kfifo_copy_to_user(struct __kfifo *fifo, void __user *to,
}
l = min(len, size - off);
- ret = copy_to_user(to, fifo->data + off, l);
+ ret = copy_to_user_partial(to, fifo->data + off, l);
if (unlikely(ret))
ret = DIV_ROUND_UP(ret + len - l, esize);
else {
- ret = copy_to_user(to + l, fifo->data, len - l);
+ ret = copy_to_user_partial(to + l, fifo->data, len - l);
if (unlikely(ret))
ret = DIV_ROUND_UP(ret, esize);
}
diff --git a/mm/kasan/kasan_test_c.c b/mm/kasan/kasan_test_c.c
index 32d06cbf6a31..a4d19fc1068a 100644
--- a/mm/kasan/kasan_test_c.c
+++ b/mm/kasan/kasan_test_c.c
@@ -2169,9 +2169,9 @@ static void copy_user_test_oob(struct kunit *test)
usermem = (char __user *)useraddr;
KUNIT_EXPECT_KASAN_FAIL(test,
- unused = copy_from_user(kmem, usermem, size + 1));
+ unused = copy_from_user_partial(kmem, usermem, size + 1));
KUNIT_EXPECT_KASAN_FAIL_READ(test,
- unused = copy_to_user(usermem, kmem, size + 1));
+ unused = copy_to_user_partial(usermem, kmem, size + 1));
KUNIT_EXPECT_KASAN_FAIL(test,
unused = __copy_from_user(kmem, usermem, size + 1));
KUNIT_EXPECT_KASAN_FAIL_READ(test,
diff --git a/mm/memory.c b/mm/memory.c
index ea6568571131..5a2f7543a2da 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -7529,7 +7529,7 @@ long copy_folio_from_user(struct folio *dst_folio,
kaddr = kmap_local_page(subpage);
if (!allow_pagefault)
pagefault_disable();
- rc = copy_from_user(kaddr, usr_src + i * PAGE_SIZE, PAGE_SIZE);
+ rc = copy_from_user_partial(kaddr, usr_src + i * PAGE_SIZE, PAGE_SIZE);
if (!allow_pagefault)
pagefault_enable();
kunmap_local(kaddr);
diff --git a/net/x25/af_x25.c b/net/x25/af_x25.c
index af8762b24039..7327c98b206a 100644
--- a/net/x25/af_x25.c
+++ b/net/x25/af_x25.c
@@ -471,7 +471,7 @@ static int x25_getsockopt(struct socket *sock, int level, int optname,
goto out;
val = test_bit(X25_Q_BIT_FLAG, &x25_sk(sk)->flags);
- rc = copy_to_user(optval, &val, len) ? -EFAULT : 0;
+ rc = copy_to_user_partial(optval, &val, len) ? -EFAULT : 0;
out:
return rc;
}
diff --git a/rust/helpers/uaccess.c b/rust/helpers/uaccess.c
index 01de4fbbcc84..710e07cd60ae 100644
--- a/rust/helpers/uaccess.c
+++ b/rust/helpers/uaccess.c
@@ -5,13 +5,13 @@
__rust_helper unsigned long
rust_helper_copy_from_user(void *to, const void __user *from, unsigned long n)
{
- return copy_from_user(to, from, n);
+ return copy_from_user_partial(to, from, n);
}
__rust_helper unsigned long
rust_helper_copy_to_user(void __user *to, const void *from, unsigned long n)
{
- return copy_to_user(to, from, n);
+ return copy_to_user_partial(to, from, n);
}
#ifndef CONFIG_ARCH_WANTS_NOINLINE_COPY_USER
diff --git a/sound/pci/emu10k1/emufx.c b/sound/pci/emu10k1/emufx.c
index 08e0556bf161..3941bf9666a9 100644
--- a/sound/pci/emu10k1/emufx.c
+++ b/sound/pci/emu10k1/emufx.c
@@ -739,10 +739,10 @@ static int copy_gctl_to_user(struct snd_emu10k1 *emu,
_dst = (struct snd_emu10k1_fx8010_control_gpr __user *)dst;
if (emu->support_tlv)
- return copy_to_user(&_dst[idx], src, sizeof(*src));
+ return copy_to_user_partial(&_dst[idx], src, sizeof(*src));
octl = (struct snd_emu10k1_fx8010_control_old_gpr __user *)dst;
- return copy_to_user(&octl[idx], src, sizeof(*octl));
+ return copy_to_user_partial(&octl[idx], src, sizeof(*octl));
}
static int copy_ctl_elem_id(const struct emu10k1_ctl_elem_id *list, int i,
diff --git a/sound/pci/rme9652/hdsp.c b/sound/pci/rme9652/hdsp.c
index 31cc2d91c8d2..d5842d8a8509 100644
--- a/sound/pci/rme9652/hdsp.c
+++ b/sound/pci/rme9652/hdsp.c
@@ -4541,7 +4541,7 @@ static int snd_hdsp_capture_release(struct snd_pcm_substream *substream)
static inline int copy_u32_le(void __user *dest, void __iomem *src)
{
u32 val = readl(src);
- return copy_to_user(dest, &val, 4);
+ return copy_to_user_partial(dest, &val, 4);
}
static inline int copy_u64_le(void __user *dest, void __iomem *src_low, void __iomem *src_high)
@@ -4551,7 +4551,7 @@ static inline int copy_u64_le(void __user *dest, void __iomem *src_low, void __i
rms_low = readl(src_low);
rms_high = readl(src_high);
rms = ((u64)rms_high << 32) | rms_low;
- return copy_to_user(dest, &rms, 8);
+ return copy_to_user_partial(dest, &rms, 8);
}
static inline int copy_u48_le(void __user *dest, void __iomem *src_low, void __iomem *src_high)
@@ -4561,7 +4561,7 @@ static inline int copy_u48_le(void __user *dest, void __iomem *src_low, void __i
rms_low = readl(src_low) & 0xffffff00;
rms_high = readl(src_high) & 0xffffff00;
rms = ((u64)rms_high << 32) | rms_low;
- return copy_to_user(dest, &rms, 8);
+ return copy_to_user_partial(dest, &rms, 8);
}
static int hdsp_9652_get_peak(struct hdsp *hdsp, struct hdsp_peak_rms __user *peak_rms)
diff --git a/sound/soc/intel/avs/probes.c b/sound/soc/intel/avs/probes.c
index 099119ad28b3..bc2871d3e18c 100644
--- a/sound/soc/intel/avs/probes.c
+++ b/sound/soc/intel/avs/probes.c
@@ -244,10 +244,10 @@ static int avs_probe_compr_copy(struct snd_soc_component *comp, struct snd_compr
n = rtd->buffer_size - offset;
if (count < n) {
- ret = copy_to_user(buf, ptr, count);
+ ret = copy_to_user_partial(buf, ptr, count);
} else {
- ret = copy_to_user(buf, ptr, n);
- ret += copy_to_user(buf + n, rtd->dma_area, count - n);
+ ret = copy_to_user_partial(buf, ptr, n);
+ ret += copy_to_user_partial(buf + n, rtd->dma_area, count - n);
}
if (ret)
diff --git a/sound/soc/sof/compress.c b/sound/soc/sof/compress.c
index 93f2376585db..d54be8a188ec 100644
--- a/sound/soc/sof/compress.c
+++ b/sound/soc/sof/compress.c
@@ -324,10 +324,10 @@ static int sof_compr_copy_playback(struct snd_compr_runtime *rtd,
n = rtd->buffer_size - offset;
if (count < n) {
- ret = copy_from_user(ptr, buf, count);
+ ret = copy_from_user_partial(ptr, buf, count);
} else {
- ret = copy_from_user(ptr, buf, n);
- ret += copy_from_user(rtd->dma_area, buf + n, count - n);
+ ret = copy_from_user_partial(ptr, buf, n);
+ ret += copy_from_user_partial(rtd->dma_area, buf + n, count - n);
}
return count - ret;
@@ -345,10 +345,10 @@ static int sof_compr_copy_capture(struct snd_compr_runtime *rtd,
n = rtd->buffer_size - offset;
if (count < n) {
- ret = copy_to_user(buf, ptr, count);
+ ret = copy_to_user_partial(buf, ptr, count);
} else {
- ret = copy_to_user(buf, ptr, n);
- ret += copy_to_user(buf + n, rtd->dma_area, count - n);
+ ret = copy_to_user_partial(buf, ptr, n);
+ ret += copy_to_user_partial(buf + n, rtd->dma_area, count - n);
}
return count - ret;
diff --git a/sound/soc/sof/sof-client-probes.c b/sound/soc/sof/sof-client-probes.c
index 124f55508159..4c5f4f016ff8 100644
--- a/sound/soc/sof/sof-client-probes.c
+++ b/sound/soc/sof/sof-client-probes.c
@@ -184,10 +184,10 @@ static int sof_probes_compr_copy(struct snd_soc_component *component,
n = rtd->buffer_size - offset;
if (count < n) {
- ret = copy_to_user(buf, ptr, count);
+ ret = copy_to_user_partial(buf, ptr, count);
} else {
- ret = copy_to_user(buf, ptr, n);
- ret += copy_to_user(buf + n, rtd->dma_area, count - n);
+ ret = copy_to_user_partial(buf, ptr, n);
+ ret += copy_to_user_partial(buf + n, rtd->dma_area, count - n);
}
if (ret)
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 6/9] uaccess: Change copy_{to/from}_user to return -EFAULT
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
Now that copy_{to/from}_user_partial() are used by callers which expect
partial copy with number of not copied bytes as return value, change
copy_{to/from}_user() to return an int, and return -EFAULT when the
copy is not complete.
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
include/linux/uaccess.h | 28 ++++++++++++++++++++++++----
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index 2d37173782b3..33b7d0f5f808 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -211,7 +211,7 @@ extern __must_check unsigned long
_copy_to_user(void __user *, const void *, unsigned long);
static __always_inline unsigned long __must_check
-copy_from_user(void *to, const void __user *from, unsigned long n)
+copy_from_user_common(void *to, const void __user *from, unsigned long n, bool partial)
{
if (!check_copy_size(to, n, false))
return n;
@@ -221,10 +221,20 @@ copy_from_user(void *to, const void __user *from, unsigned long n)
return _inline_copy_from_user(to, from, n);
}
-#define copy_from_user_partial copy_from_user
+static __always_inline unsigned long __must_check
+copy_from_user_partial(void *to, const void __user *from, unsigned long n)
+{
+ return copy_from_user_common(to, from, n, true);
+}
+
+static __always_inline int __must_check
+copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+ return copy_from_user_common(to, from, n, false) ? -EFAULT : 0;
+}
static __always_inline unsigned long __must_check
-copy_to_user(void __user *to, const void *from, unsigned long n)
+copy_to_user_common(void __user *to, const void *from, unsigned long n, bool partial)
{
if (!check_copy_size(from, n, true))
return n;
@@ -235,7 +245,17 @@ copy_to_user(void __user *to, const void *from, unsigned long n)
return _inline_copy_to_user(to, from, n);
}
-#define copy_to_user_partial copy_to_user
+static __always_inline unsigned long __must_check
+copy_to_user_partial(void __user *to, const void *from, unsigned long n)
+{
+ return copy_to_user_common(to, from, n, true);
+}
+
+static __always_inline int __must_check
+copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+ return copy_to_user_common(to, from, n, false) ? -EFAULT : 0;
+}
#ifndef copy_mc_to_kernel
/*
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 7/9] x86: Add unsafe_copy_from_user()
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
At the time being, x86 and arm64 are missing unsafe_copy_from_user().
Add it.
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
arch/x86/include/asm/uaccess.h | 29 ++++++++++++++++++++++++-----
1 file changed, 24 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index 3a0dd3c2b233..10c458ffa399 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -598,7 +598,7 @@ _label: \
* We want the unsafe accessors to always be inlined and use
* the error labels - thus the macro games.
*/
-#define unsafe_copy_loop(dst, src, len, type, label) \
+#define unsafe_put_loop(dst, src, len, type, label) \
while (len >= sizeof(type)) { \
unsafe_put_user(*(type *)(src),(type __user *)(dst),label); \
dst += sizeof(type); \
@@ -611,10 +611,29 @@ do { \
char __user *__ucu_dst = (_dst); \
const char *__ucu_src = (_src); \
size_t __ucu_len = (_len); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u64, label); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u32, label); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u16, label); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u64, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u32, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u16, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \
+} while (0)
+
+#define unsafe_get_loop(dst, src, len, type, label) \
+ while (len >= sizeof(type)) { \
+ unsafe_get_user(*(type __user *)(src),(type *)(dst),label); \
+ dst += sizeof(type); \
+ src += sizeof(type); \
+ len -= sizeof(type); \
+ }
+
+#define unsafe_copy_from_user(_dst,_src,_len,label) \
+do { \
+ char *__ucu_dst = (_dst); \
+ const char __user *__ucu_src = (_src); \
+ size_t __ucu_len = (_len); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u64, label); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u32, label); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u16, label); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \
} while (0)
#ifdef CONFIG_CC_HAS_ASM_GOTO_OUTPUT
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 8/9] arm64: Add unsafe_copy_from_user()
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
At the time being, x86 and arm64 are missing unsafe_copy_from_user().
Add it.
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
arch/arm64/include/asm/uaccess.h | 29 ++++++++++++++++++++++++-----
1 file changed, 24 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index 1e20ec91b56f..adfdb52cd82b 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -437,7 +437,7 @@ static inline void user_access_restore(unsigned long enabled) { }
* We want the unsafe accessors to always be inlined and use
* the error labels - thus the macro games.
*/
-#define unsafe_copy_loop(dst, src, len, type, label) \
+#define unsafe_put_loop(dst, src, len, type, label) \
while (len >= sizeof(type)) { \
unsafe_put_user(*(type *)(src),(type __user *)(dst),label); \
dst += sizeof(type); \
@@ -450,10 +450,29 @@ do { \
char __user *__ucu_dst = (_dst); \
const char *__ucu_src = (_src); \
size_t __ucu_len = (_len); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u64, label); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u32, label); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u16, label); \
- unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u64, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u32, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u16, label); \
+ unsafe_put_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \
+} while (0)
+
+#define unsafe_get_loop(dst, src, len, type, label) \
+ while (len >= sizeof(type)) { \
+ unsafe_get_user(*(type __user *)(src),(type *)(dst),label); \
+ dst += sizeof(type); \
+ src += sizeof(type); \
+ len -= sizeof(type); \
+ }
+
+#define unsafe_copy_from_user(_dst,_src,_len,label) \
+do { \
+ char *__ucu_dst = (_dst); \
+ const char __user *__ucu_src = (_src); \
+ size_t __ucu_len = (_len); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u64, label); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u32, label); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u16, label); \
+ unsafe_get_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \
} while (0)
extern unsigned long __must_check __arch_clear_user(void __user *to, unsigned long n);
--
2.49.0
^ permalink raw reply related
* [RFC PATCH v1 9/9] uaccess: Convert small fixed size copy_{to/from}_user() to scoped user access
From: Christophe Leroy (CS GROUP) @ 2026-04-27 17:13 UTC (permalink / raw)
To: Yury Norov, Andrew Morton, Linus Torvalds, David Laight,
Thomas Gleixner
Cc: Christophe Leroy (CS GROUP), linux-alpha, linux-kernel,
linux-snps-arc, linux-arm-kernel, linux-mips, linuxppc-dev, kvm,
linux-riscv, linux-s390, sparclinux, linux-um, dmaengine,
linux-efi, linux-fsi, amd-gfx, dri-devel, intel-gfx, linux-wpan,
netdev, linux-wireless, linux-spi, linux-media, linux-staging,
linux-serial, linux-usb, xen-devel, linux-fsdevel, ocfs2-devel,
bpf, kasan-dev, linux-mm, linux-x25, rust-for-linux, linux-sound,
sound-open-firmware, linux-csky, linux-hexagon, loongarch,
linux-m68k, linux-openrisc, linux-parisc, linux-sh, linux-arch
In-Reply-To: <cover.1777306795.git.chleroy@kernel.org>
copy_{to/from}_user() is a heavy function optimised for copy of large
blocs of memory between user and kernel space.
When the number of bytes to be copied is known at build time and small,
using scoped user access removes the burden of that optimisation.
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
---
include/linux/uaccess.h | 47 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index 33b7d0f5f808..3ac544527af2 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -50,6 +50,8 @@
#define mask_user_address(src) (src)
#endif
+#define SMALL_COPY_USER 64
+
/*
* Architectures should provide two primitives (raw_copy_{to,from}_user())
* and get rid of their private instances of copy_{to,from}_user() and
@@ -191,6 +193,9 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n)
return res;
}
+static __always_inline __must_check unsigned long
+_small_copy_from_user(void *to, const void __user *from, unsigned long n);
+
extern __must_check unsigned long
_copy_from_user(void *, const void __user *, unsigned long);
@@ -207,6 +212,9 @@ _inline_copy_to_user(void __user *to, const void *from, unsigned long n)
return n;
}
+static __always_inline __must_check unsigned long
+_small_copy_to_user(void __user *to, const void *from, unsigned long n);
+
extern __must_check unsigned long
_copy_to_user(void __user *, const void *, unsigned long);
@@ -215,6 +223,8 @@ copy_from_user_common(void *to, const void __user *from, unsigned long n, bool p
{
if (!check_copy_size(to, n, false))
return n;
+ if (!partial && __builtin_constant_p(n) && n <= SMALL_COPY_USER)
+ return _small_copy_from_user(to, from, n);
if (IS_ENABLED(ARCH_WANTS_NOINLINE_COPY_USER))
return _copy_from_user(to, from, n);
else
@@ -239,6 +249,8 @@ copy_to_user_common(void __user *to, const void *from, unsigned long n, bool par
if (!check_copy_size(from, n, true))
return n;
+ if (!partial && __builtin_constant_p(n) && n <= SMALL_COPY_USER)
+ return _small_copy_to_user(to, from, n);
if (IS_ENABLED(ARCH_WANTS_NOINLINE_COPY_USER))
return _copy_to_user(to, from, n);
else
@@ -838,6 +850,41 @@ for (bool done = false; !done; done = true) \
#define scoped_user_rw_access(uptr, elbl) \
scoped_user_rw_access_size(uptr, sizeof(*(uptr)), elbl)
+static __always_inline __must_check unsigned long
+_small_copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+ might_fault();
+ instrument_copy_from_user_before(to, from, n);
+ scoped_user_read_access_size(from, n, failed) {
+ /*
+ * Ensure that bad access_ok() speculation will not lead
+ * to nasty side effects *after* the copy is finished:
+ */
+ if (!can_do_masked_user_access())
+ barrier_nospec();
+ unsafe_copy_from_user(to, from, n, failed);
+ }
+ instrument_copy_from_user_after(to, from, n, 0);
+ return 0;
+failed:
+ instrument_copy_from_user_after(to, from, n, n);
+ return n;
+}
+
+static __always_inline __must_check unsigned long
+_small_copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+ might_fault();
+ if (should_fail_usercopy())
+ return n;
+ instrument_copy_to_user(to, from, n);
+ scoped_user_write_access_size(to, n, failed)
+ unsafe_copy_to_user(to, from, n, failed);
+ return 0;
+failed:
+ return n;
+}
+
/**
* get_user_inline - Read user data inlined
* @val: The variable to store the value read from user memory
--
2.49.0
^ permalink raw reply related
* [syzbot] [lvs?] UBSAN: shift-out-of-bounds in ip_vs_rht_desired_size
From: syzbot @ 2026-04-27 17:27 UTC (permalink / raw)
To: coreteam, davem, edumazet, fw, horms, ja, kuba, linux-kernel,
lvs-devel, netdev, netfilter-devel, pabeni, pablo, phil,
syzkaller-bugs
Hello,
syzbot found the following issue on:
HEAD commit: e728258debd5 Merge tag 'net-7.1-rc1' of git://git.kernel.o..
git tree: net-next
console output: https://syzkaller.appspot.com/x/log.txt?x=169022ce580000
kernel config: https://syzkaller.appspot.com/x/.config?x=ca77bfc4078c8193
dashboard link: https://syzkaller.appspot.com/bug?extid=217f1db9c791e27fe54a
compiler: Debian clang version 21.1.8 (++20251221033036+2078da43e25a-1~exp1~20251221153213.50), Debian LLD 21.1.8
Unfortunately, I don't have any reproducer for this issue yet.
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/24195bde5d1d/disk-e728258d.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/78131d1b0e14/vmlinux-e728258d.xz
kernel image: https://storage.googleapis.com/syzbot-assets/836d0dd78c10/bzImage-e728258d.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+217f1db9c791e27fe54a@syzkaller.appspotmail.com
wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge)
------------[ cut here ]------------
UBSAN: shift-out-of-bounds in ./include/linux/log2.h:57:13
shift exponent 64 is too large for 64-bit type 'unsigned long'
CPU: 1 UID: 0 PID: 77 Comm: kworker/u8:4 Not tainted syzkaller #0 PREEMPT(full)
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/18/2026
Workqueue: events_unbound conn_resize_work_handler
Call Trace:
<TASK>
dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120
ubsan_epilogue+0xa/0x30 lib/ubsan.c:233
__ubsan_handle_shift_out_of_bounds+0x385/0x410 lib/ubsan.c:494
__roundup_pow_of_two include/linux/log2.h:57 [inline]
ip_vs_rht_desired_size+0x2cf/0x410 net/netfilter/ipvs/ip_vs_core.c:240
ip_vs_conn_desired_size net/netfilter/ipvs/ip_vs_conn.c:765 [inline]
conn_resize_work_handler+0x1b6/0x14c0 net/netfilter/ipvs/ip_vs_conn.c:822
process_one_work kernel/workqueue.c:3302 [inline]
process_scheduled_works+0xb5d/0x1860 kernel/workqueue.c:3385
worker_thread+0xa53/0xfc0 kernel/workqueue.c:3466
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
</TASK>
---[ end trace ]---
Kernel panic - not syncing: UBSAN: panic_on_warn set ...
CPU: 1 UID: 0 PID: 77 Comm: kworker/u8:4 Not tainted syzkaller #0 PREEMPT(full)
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/18/2026
Workqueue: events_unbound conn_resize_work_handler
Call Trace:
<TASK>
vpanic+0x56c/0xa60 kernel/panic.c:650
panic+0xc5/0xd0 kernel/panic.c:787
check_panic_on_warn+0x89/0xb0 kernel/panic.c:524
__ubsan_handle_shift_out_of_bounds+0x385/0x410 lib/ubsan.c:494
__roundup_pow_of_two include/linux/log2.h:57 [inline]
ip_vs_rht_desired_size+0x2cf/0x410 net/netfilter/ipvs/ip_vs_core.c:240
ip_vs_conn_desired_size net/netfilter/ipvs/ip_vs_conn.c:765 [inline]
conn_resize_work_handler+0x1b6/0x14c0 net/netfilter/ipvs/ip_vs_conn.c:822
process_one_work kernel/workqueue.c:3302 [inline]
process_scheduled_works+0xb5d/0x1860 kernel/workqueue.c:3385
worker_thread+0xa53/0xfc0 kernel/workqueue.c:3466
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
</TASK>
Kernel Offset: disabled
Rebooting in 86400 seconds..
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
^ permalink raw reply
* [PATCH net-next v2 0/5] Reimplement TCP-AO using crypto library
From: Eric Biggers @ 2026-04-27 17:27 UTC (permalink / raw)
To: netdev
Cc: linux-crypto, linux-kernel, Eric Dumazet, Neal Cardwell,
Kuniyuki Iwashima, David S . Miller, David Ahern, Jakub Kicinski,
Paolo Abeni, Simon Horman, Ard Biesheuvel, Jason A . Donenfeld,
Herbert Xu, Dmitry Safonov, Eric Biggers
This series can also be retrieved from:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git tcp-ao-v2
This series is targeting net-next for 7.2. To make this series
self-contained in the networking code, I dropped the patches that remove
support for transformation cloning from the crypto API, which is a
further negative 275-line cleanup and optimization this series enables.
That will be done as a follow-up, either through the crypto tree for
7.3, or still through net-next for 7.2 at maintainer preference.
This series refactors the TCP-AO (TCP Authentication Option) code to do
MAC and KDF computations using lib/crypto/ instead of crypto_ahash.
This greatly simplifies the code and makes it much more efficient. The
entire tcp_sigpool mechanism becomes unnecessary and is removed, as the
problems it was designed to solve don't exist with the library APIs.
The crypto API's support for crypto transformation cloning also becomes
unnecessary and will be removed in follow-up patches. Note that as part
of that, we'll be able to roll back the addition of the reference count
to crypto_tfm, which had regressed performance for all crypto API users.
To make this simplification and optimization possible, this series also
updates the TCP-AO code to support a specific set of algorithms, rather
than arbitrary algorithms that don't make sense and are very likely not
being used, e.g. CRC-32 and HMAC-MD5.
Specifically, this series retains the support for AES-128-CMAC,
HMAC-SHA1, and HMAC-SHA256. AES-128-CMAC and HMAC-SHA1 are the only
algorithms that are actually standardized for use in TCP-AO, while
HMAC-SHA256 makes sense to continue supporting as a Linux extension. Of
course, other algorithms can still be (re-)added later if ever needed.
It's worth noting that TCP-AO MACs are limited to 20 bytes by the TCP
options space, which limits the benefit of further algorithm upgrades.
This series passes the tcp_ao selftests
(sudo make -C tools/testing/selftests/net/tcp_ao/ run_tests).
To get a sense for how much more efficient this makes the TCP-AO code,
here's a microbenchmark for tcp_ao_hash_skb() with skb->len == 128:
Algorithm Avg cycles (before) Avg cycles (after)
--------- ------------------- ------------------
HMAC-SHA1 3319 1256
HMAC-SHA256 3311 1344
AES-128-CMAC 2720 1107
Changed in v2:
- Rebased onto v7.1-rc1.
- Added Ard's Reviewed-by.
- Dropped patches that clean up things in the crypto/ directory, as
mentioned above. They'll be sent separately.
- Added some mentions of the MAC length being limited by the TCP
options space.
- Removed unnecessary explicit assignment of values to enums.
Eric Biggers (5):
net/tcp-ao: Drop support for most non-RFC-specified algorithms
net/tcp-ao: Use crypto library API instead of crypto_ahash
net/tcp-ao: Use stack-allocated MAC and traffic_key buffers
net/tcp-ao: Return void from functions that can no longer fail
net/tcp: Remove tcp_sigpool
include/net/tcp.h | 42 +-
include/net/tcp_ao.h | 74 +-
net/ipv4/Kconfig | 8 +-
net/ipv4/Makefile | 1 -
net/ipv4/tcp_ao.c | 677 +++++++++---------
net/ipv4/tcp_output.c | 10 +-
net/ipv4/tcp_sigpool.c | 366 ----------
net/ipv6/tcp_ao.c | 139 ++--
tools/testing/selftests/net/tcp_ao/config | 4 -
.../selftests/net/tcp_ao/key-management.c | 41 +-
10 files changed, 440 insertions(+), 922 deletions(-)
delete mode 100644 net/ipv4/tcp_sigpool.c
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
--
2.54.0
^ permalink raw reply
* [PATCH net-next v2 1/5] net/tcp-ao: Drop support for most non-RFC-specified algorithms
From: Eric Biggers @ 2026-04-27 17:27 UTC (permalink / raw)
To: netdev
Cc: linux-crypto, linux-kernel, Eric Dumazet, Neal Cardwell,
Kuniyuki Iwashima, David S . Miller, David Ahern, Jakub Kicinski,
Paolo Abeni, Simon Horman, Ard Biesheuvel, Jason A . Donenfeld,
Herbert Xu, Dmitry Safonov, Eric Biggers
In-Reply-To: <20260427172727.9310-1-ebiggers@kernel.org>
RFC 5926 (https://datatracker.ietf.org/doc/html/rfc5926) specifies the
use of AES-128-CMAC and HMAC-SHA1 with TCP-AO. This includes a
specification for how traffic keys shall be derived for each algorithm.
Support for any other algorithms with TCP-AO isn't standardized, though
an expired Internet Draft (a work-in-progress document, not a standard)
from 2019 does propose adding HMAC-SHA256 support:
https://datatracker.ietf.org/doc/html/draft-nayak-tcp-sha2-03
Since both documents specify the KDF for each algorithm individually, it
isn't necessarily clear how any other algorithm should be integrated.
Nevertheless, the Linux implementation of TCP-AO allows userspace to
specify the MAC algorithm as a string tcp_ao_add::alg_name naming either
"cmac(aes128)" or an arbitrary algorithm in the crypto_ahash API. The
set of valid strings is undocumented. The implementation assumes that
"cmac(aes128)" is the only algorithm that requires an entropy extraction
step and that all algorithms accept keys with length equal to the
untruncated MAC; thus, arbitrary HMAC algorithms probably do work, but
some other MAC algorithms like AES-256-CMAC have never actually worked.
Unfortunately, this undocumented string allows many obsolete, insecure,
or redundant algorithms. For example, "hmac(md5)" and the
non-cryptographic "crc32" are accepted. It also ties the implementation
to crypto_ahash and requires that most memory be dynamically allocated,
making the implementation unnecessarily complex and inefficient. Still
furthermore, this implementation requires the crypto API to support
"transformation cloning", whose only user is this feature.
Fortunately, it's very likely that only a few algorithms are actually
used in practice. Let's restrict the set of allowed algorithms to
"cmac(aes128)" (or "cmac(aes)" with keylen=16), "hmac(sha1)", and
"hmac(sha256)". The first two are the actually standard ones, while
HMAC-SHA256 seems like a reasonable algorithm to continue supporting as
a Linux extension, considering the Internet Draft for it and the fact
that SHA-256 is the usual choice of upgrade from the outdated SHA-1.
If any other algorithm ever turns out to be needed, e.g. HMAC-SHA512, it
can of course be (re-)added in library form. However, note that the TCP
options space limits TCP-AO MACs to 20 bytes (160 bits) anyway, which
limits the potential benefit of any further upgrade to the algorithm.
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
net/ipv4/tcp_ao.c | 4 ++
tools/testing/selftests/net/tcp_ao/config | 1 -
.../selftests/net/tcp_ao/key-management.c | 41 ++-----------------
3 files changed, 7 insertions(+), 39 deletions(-)
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index a97cdf3e6af4..b21bd69b4e82 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -1561,10 +1561,14 @@ static struct tcp_ao_key *tcp_ao_key_alloc(struct sock *sk,
cmd->alg_name[ARRAY_SIZE(cmd->alg_name) - 1] = '\0';
/* RFC5926, 3.1.1.2. KDF_AES_128_CMAC */
if (!strcmp("cmac(aes128)", algo))
algo = "cmac(aes)";
+ else if (strcmp("hmac(sha1)", algo) &&
+ strcmp("hmac(sha256)", algo) &&
+ (strcmp("cmac(aes)", algo) || cmd->keylen != 16))
+ return ERR_PTR(-ENOENT);
/* Full TCP header (th->doff << 2) should fit into scratch area,
* see tcp_ao_hash_header().
*/
pool_id = tcp_sigpool_alloc_ahash(algo, 60);
diff --git a/tools/testing/selftests/net/tcp_ao/config b/tools/testing/selftests/net/tcp_ao/config
index f22148512365..47228a7d0b90 100644
--- a/tools/testing/selftests/net/tcp_ao/config
+++ b/tools/testing/selftests/net/tcp_ao/config
@@ -1,8 +1,7 @@
CONFIG_CRYPTO_CMAC=y
CONFIG_CRYPTO_HMAC=y
-CONFIG_CRYPTO_RMD160=y
CONFIG_CRYPTO_SHA1=y
CONFIG_IPV6=y
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_NET_L3_MASTER_DEV=y
CONFIG_NET_VRF=y
diff --git a/tools/testing/selftests/net/tcp_ao/key-management.c b/tools/testing/selftests/net/tcp_ao/key-management.c
index 69d9a7a05d5c..d86bb380b79f 100644
--- a/tools/testing/selftests/net/tcp_ao/key-management.c
+++ b/tools/testing/selftests/net/tcp_ao/key-management.c
@@ -378,35 +378,10 @@ static void check_listen_socket(void)
this_ip_dest, DEFAULT_TEST_PREFIX,
false, true, 20, 10, FAULT_CURRNEXT);
close(sk);
}
-static const char *fips_fpath = "/proc/sys/crypto/fips_enabled";
-static bool is_fips_enabled(void)
-{
- static int fips_checked = -1;
- FILE *fenabled;
- int enabled;
-
- if (fips_checked >= 0)
- return !!fips_checked;
- if (access(fips_fpath, R_OK)) {
- if (errno != ENOENT)
- test_error("Can't open %s", fips_fpath);
- fips_checked = 0;
- return false;
- }
- fenabled = fopen(fips_fpath, "r");
- if (!fenabled)
- test_error("Can't open %s", fips_fpath);
- if (fscanf(fenabled, "%d", &enabled) != 1)
- test_error("Can't read from %s", fips_fpath);
- fclose(fenabled);
- fips_checked = !!enabled;
- return !!fips_checked;
-}
-
struct test_key {
char password[TCP_AO_MAXKEYLEN];
const char *alg;
unsigned int len;
uint8_t client_keyid;
@@ -428,18 +403,11 @@ struct key_collection {
};
static struct key_collection collection;
#define TEST_MAX_MACLEN 16
-const char *test_algos[] = {
- "cmac(aes128)",
- "hmac(sha1)", "hmac(sha512)", "hmac(sha384)", "hmac(sha256)",
- "hmac(sha224)", "hmac(sha3-512)",
- /* only if !CONFIG_FIPS */
-#define TEST_NON_FIPS_ALGOS 2
- "hmac(rmd160)", "hmac(md5)"
-};
+const char *test_algos[] = { "cmac(aes128)", "hmac(sha1)", "hmac(sha256)" };
const unsigned int test_maclens[] = { 1, 4, 12, 16 };
#define MACLEN_SHIFT 2
#define ALGOS_SHIFT 4
static unsigned int make_mask(unsigned int shift, unsigned int prev_shift)
@@ -450,11 +418,11 @@ static unsigned int make_mask(unsigned int shift, unsigned int prev_shift)
}
static void init_key_in_collection(unsigned int index, bool randomized)
{
struct test_key *key = &collection.keys[index];
- unsigned int algos_nr, algos_index;
+ unsigned int algos_index;
/* Same for randomized and non-randomized test flows */
key->client_keyid = index;
key->server_keyid = 127 + index;
key->matches_client = 1;
@@ -472,14 +440,11 @@ static void init_key_in_collection(unsigned int index, bool randomized)
unsigned int shift = MACLEN_SHIFT;
key->maclen = test_maclens[index & make_mask(shift, 0)];
algos_index = index & make_mask(ALGOS_SHIFT, shift);
}
- algos_nr = ARRAY_SIZE(test_algos);
- if (is_fips_enabled())
- algos_nr -= TEST_NON_FIPS_ALGOS;
- key->alg = test_algos[algos_index % algos_nr];
+ key->alg = test_algos[algos_index % ARRAY_SIZE(test_algos)];
}
static int init_default_key_collection(unsigned int nr_keys, bool randomized)
{
size_t key_sz = sizeof(collection.keys[0]);
--
2.54.0
^ permalink raw reply related
* [PATCH net-next v2 2/5] net/tcp-ao: Use crypto library API instead of crypto_ahash
From: Eric Biggers @ 2026-04-27 17:27 UTC (permalink / raw)
To: netdev
Cc: linux-crypto, linux-kernel, Eric Dumazet, Neal Cardwell,
Kuniyuki Iwashima, David S . Miller, David Ahern, Jakub Kicinski,
Paolo Abeni, Simon Horman, Ard Biesheuvel, Jason A . Donenfeld,
Herbert Xu, Dmitry Safonov, Eric Biggers
In-Reply-To: <20260427172727.9310-1-ebiggers@kernel.org>
Currently the kernel's TCP-AO implementation does the MAC and KDF
computations using the crypto_ahash API. This API is inefficient and
difficult to use, and it has required extensive workarounds in the form
of per-CPU preallocated objects (tcp_sigpool) to work at all.
Let's use lib/crypto/ instead. This means switching to straightforward
stack-allocated structures, virtually addressed buffers, and direct
function calls. It also means removing quite a bit of error handling.
This makes TCP-AO quite a bit faster.
This also enables many additional cleanups, which later commits will
handle: removing tcp-sigpool, removing support for crypto_tfm cloning,
removing more error handling, and replacing more dynamically-allocated
buffers with stack buffers based on the now-statically-known limits.
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
include/net/tcp_ao.h | 32 +-
net/ipv4/Kconfig | 5 +-
net/ipv4/tcp_ao.c | 523 +++++++++++-----------
net/ipv6/tcp_ao.c | 63 ++-
tools/testing/selftests/net/tcp_ao/config | 3 -
5 files changed, 309 insertions(+), 317 deletions(-)
diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 1e9e27d6e06b..20997aef3b0d 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -1,11 +1,10 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
#ifndef _TCP_AO_H
#define _TCP_AO_H
-#define TCP_AO_KEY_ALIGN 1
-#define __tcp_ao_key_align __aligned(TCP_AO_KEY_ALIGN)
+#include <crypto/sha2.h> /* for SHA256_DIGEST_SIZE */
union tcp_ao_addr {
struct in_addr a4;
#if IS_ENABLED(CONFIG_IPV6)
struct in6_addr a6;
@@ -30,15 +29,31 @@ struct tcp_ao_counters {
atomic64_t key_not_found;
atomic64_t ao_required;
atomic64_t dropped_icmp;
};
+enum tcp_ao_algo_id {
+ TCP_AO_ALGO_HMAC_SHA1 = 1, /* specified by RFC 5926 */
+ TCP_AO_ALGO_HMAC_SHA256, /* Linux extension */
+ TCP_AO_ALGO_AES_128_CMAC, /* specified by RFC 5926 */
+};
+
+/*
+ * This is the maximum untruncated MAC length, in bytes. Note that the MACs
+ * actually get truncated to 20 or fewer bytes to fit in the TCP options space.
+ */
+#define TCP_AO_MAX_MAC_LEN SHA256_DIGEST_SIZE
+
+#define TCP_AO_MAX_TRAFFIC_KEY_LEN SHA256_DIGEST_SIZE
+
+struct tcp_ao_mac_ctx;
+
struct tcp_ao_key {
struct hlist_node node;
union tcp_ao_addr addr;
- u8 key[TCP_AO_MAXKEYLEN] __tcp_ao_key_align;
- unsigned int tcp_sigpool_id;
+ u8 key[TCP_AO_MAXKEYLEN];
+ enum tcp_ao_algo_id algo;
unsigned int digest_size;
int l3index;
u8 prefixlen;
u8 family;
u8 keylen;
@@ -166,18 +181,19 @@ struct tcp6_ao_context {
__be16 dport;
__be32 sisn;
__be32 disn;
};
-struct tcp_sigpool;
/* Established states are fast-path and there always is current_key/rnext_key */
#define TCP_AO_ESTABLISHED (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2 | \
TCPF_CLOSE_WAIT | TCPF_LAST_ACK | TCPF_CLOSING)
int tcp_ao_transmit_skb(struct sock *sk, struct sk_buff *skb,
struct tcp_ao_key *key, struct tcphdr *th,
__u8 *hash_location);
+void tcp_ao_mac_update(struct tcp_ao_mac_ctx *mac_ctx, const void *data,
+ size_t data_len);
int tcp_ao_hash_skb(unsigned short int family,
char *ao_hash, struct tcp_ao_key *key,
const struct sock *sk, const struct sk_buff *skb,
const u8 *tkey, int hash_offset, u32 sne);
int tcp_parse_ao(struct sock *sk, int cmd, unsigned short int family,
@@ -186,12 +202,12 @@ struct tcp_ao_key *tcp_ao_established_key(const struct sock *sk,
struct tcp_ao_info *ao,
int sndid, int rcvid);
int tcp_ao_copy_all_matching(const struct sock *sk, struct sock *newsk,
struct request_sock *req, struct sk_buff *skb,
int family);
-int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
- unsigned int len, struct tcp_sigpool *hp);
+void tcp_ao_calc_traffic_key(const struct tcp_ao_key *mkt, u8 *traffic_key,
+ const void *input, unsigned int input_len);
void tcp_ao_destroy_sock(struct sock *sk, bool twsk);
void tcp_ao_time_wait(struct tcp_timewait_sock *tcptw, struct tcp_sock *tp);
bool tcp_ao_ignore_icmp(const struct sock *sk, int family, int type, int code);
int tcp_ao_get_mkts(struct sock *sk, sockptr_t optval, sockptr_t optlen);
int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockptr_t optlen);
@@ -232,11 +248,11 @@ struct tcp_ao_key *tcp_v4_ao_lookup_rsk(const struct sock *sk,
int sndid, int rcvid);
int tcp_v4_ao_hash_skb(char *ao_hash, struct tcp_ao_key *key,
const struct sock *sk, const struct sk_buff *skb,
const u8 *tkey, int hash_offset, u32 sne);
/* ipv6 specific functions */
-int tcp_v6_ao_hash_pseudoheader(struct tcp_sigpool *hp,
+int tcp_v6_ao_hash_pseudoheader(struct tcp_ao_mac_ctx *mac_ctx,
const struct in6_addr *daddr,
const struct in6_addr *saddr, int nbytes);
int tcp_v6_ao_calc_key_skb(struct tcp_ao_key *mkt, u8 *key,
const struct sk_buff *skb, __be32 sisn, __be32 disn);
int tcp_v6_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig
index 21e5164e30db..77b053b445a0 100644
--- a/net/ipv4/Kconfig
+++ b/net/ipv4/Kconfig
@@ -744,13 +744,14 @@ config DEFAULT_TCP_CONG
config TCP_SIGPOOL
tristate
config TCP_AO
bool "TCP: Authentication Option (RFC5925)"
- select CRYPTO
+ select CRYPTO_LIB_AES_CBC_MACS
+ select CRYPTO_LIB_SHA1
+ select CRYPTO_LIB_SHA256
select CRYPTO_LIB_UTILS
- select TCP_SIGPOOL
depends on 64BIT # seq-number extension needs WRITE_ONCE(u64)
help
TCP-AO specifies the use of stronger Message Authentication Codes (MACs),
protects against replays for long-lived TCP connections, and
provides more details on the association of security with TCP
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index b21bd69b4e82..0d24cbd66c9a 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -7,11 +7,13 @@
* Francesco Ruggeri <fruggeri@arista.com>
* Salam Noureddine <noureddine@arista.com>
*/
#define pr_fmt(fmt) "TCP: " fmt
-#include <crypto/hash.h>
+#include <crypto/aes-cbc-macs.h>
+#include <crypto/sha1.h>
+#include <crypto/sha2.h>
#include <crypto/utils.h>
#include <linux/inetdevice.h>
#include <linux/tcp.h>
#include <net/tcp.h>
@@ -19,36 +21,137 @@
#include <net/icmp.h>
#include <trace/events/tcp.h>
DEFINE_STATIC_KEY_DEFERRED_FALSE(tcp_ao_needed, HZ);
-int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
- unsigned int len, struct tcp_sigpool *hp)
-{
- struct scatterlist sg;
- int ret;
+static const struct tcp_ao_algo {
+ const char *name;
+ unsigned int digest_size;
+} tcp_ao_algos[] = {
+ [TCP_AO_ALGO_HMAC_SHA1] = {
+ .name = "hmac(sha1)",
+ .digest_size = SHA1_DIGEST_SIZE,
+ },
+ [TCP_AO_ALGO_HMAC_SHA256] = {
+ .name = "hmac(sha256)",
+ .digest_size = SHA256_DIGEST_SIZE,
+ },
+ [TCP_AO_ALGO_AES_128_CMAC] = {
+ .name = "cmac(aes128)",
+ .digest_size = AES_BLOCK_SIZE, /* same as AES_KEYSIZE_128 */
+ },
+};
+
+struct tcp_ao_mac_ctx {
+ enum tcp_ao_algo_id algo;
+ union {
+ struct hmac_sha1_ctx hmac_sha1;
+ struct hmac_sha256_ctx hmac_sha256;
+ struct {
+ struct aes_cmac_key key;
+ struct aes_cmac_ctx ctx;
+ } aes_cmac;
+ };
+};
+
+static const struct tcp_ao_algo *tcp_ao_find_algo(const char *name)
+{
+ for (size_t i = 0; i < ARRAY_SIZE(tcp_ao_algos); i++) {
+ const struct tcp_ao_algo *algo = &tcp_ao_algos[i];
+
+ if (!algo->name)
+ continue;
+ if (WARN_ON_ONCE(algo->digest_size > TCP_AO_MAX_MAC_LEN ||
+ algo->digest_size >
+ TCP_AO_MAX_TRAFFIC_KEY_LEN))
+ continue;
+ if (strcmp(name, algo->name) == 0)
+ return algo;
+ }
+ return NULL;
+}
- if (crypto_ahash_setkey(crypto_ahash_reqtfm(hp->req),
- mkt->key, mkt->keylen))
- goto clear_hash;
+static void tcp_ao_mac_init(struct tcp_ao_mac_ctx *mac_ctx,
+ enum tcp_ao_algo_id algo, const u8 *traffic_key)
+{
+ mac_ctx->algo = algo;
+ switch (mac_ctx->algo) {
+ case TCP_AO_ALGO_HMAC_SHA1:
+ hmac_sha1_init_usingrawkey(&mac_ctx->hmac_sha1, traffic_key,
+ SHA1_DIGEST_SIZE);
+ return;
+ case TCP_AO_ALGO_HMAC_SHA256:
+ hmac_sha256_init_usingrawkey(&mac_ctx->hmac_sha256, traffic_key,
+ SHA256_DIGEST_SIZE);
+ return;
+ case TCP_AO_ALGO_AES_128_CMAC:
+ aes_cmac_preparekey(&mac_ctx->aes_cmac.key, traffic_key,
+ AES_KEYSIZE_128);
+ aes_cmac_init(&mac_ctx->aes_cmac.ctx, &mac_ctx->aes_cmac.key);
+ return;
+ default:
+ WARN_ON_ONCE(1); /* algo was validated earlier. */
+ }
+}
- ret = crypto_ahash_init(hp->req);
- if (ret)
- goto clear_hash;
+void tcp_ao_mac_update(struct tcp_ao_mac_ctx *mac_ctx, const void *data,
+ size_t data_len)
+{
+ switch (mac_ctx->algo) {
+ case TCP_AO_ALGO_HMAC_SHA1:
+ hmac_sha1_update(&mac_ctx->hmac_sha1, data, data_len);
+ return;
+ case TCP_AO_ALGO_HMAC_SHA256:
+ hmac_sha256_update(&mac_ctx->hmac_sha256, data, data_len);
+ return;
+ case TCP_AO_ALGO_AES_128_CMAC:
+ aes_cmac_update(&mac_ctx->aes_cmac.ctx, data, data_len);
+ return;
+ default:
+ WARN_ON_ONCE(1); /* algo was validated earlier. */
+ }
+}
- sg_init_one(&sg, ctx, len);
- ahash_request_set_crypt(hp->req, &sg, key, len);
- crypto_ahash_update(hp->req);
+static void tcp_ao_mac_final(struct tcp_ao_mac_ctx *mac_ctx, u8 *out)
+{
+ switch (mac_ctx->algo) {
+ case TCP_AO_ALGO_HMAC_SHA1:
+ hmac_sha1_final(&mac_ctx->hmac_sha1, out);
+ return;
+ case TCP_AO_ALGO_HMAC_SHA256:
+ hmac_sha256_final(&mac_ctx->hmac_sha256, out);
+ return;
+ case TCP_AO_ALGO_AES_128_CMAC:
+ aes_cmac_final(&mac_ctx->aes_cmac.ctx, out);
+ return;
+ default:
+ WARN_ON_ONCE(1); /* algo was validated earlier. */
+ }
+}
- ret = crypto_ahash_final(hp->req);
- if (ret)
- goto clear_hash;
+void tcp_ao_calc_traffic_key(const struct tcp_ao_key *mkt, u8 *traffic_key,
+ const void *input, unsigned int input_len)
+{
+ switch (mkt->algo) {
+ case TCP_AO_ALGO_HMAC_SHA1:
+ hmac_sha1_usingrawkey(mkt->key, mkt->keylen, input, input_len,
+ traffic_key);
+ return;
+ case TCP_AO_ALGO_HMAC_SHA256:
+ hmac_sha256_usingrawkey(mkt->key, mkt->keylen, input, input_len,
+ traffic_key);
+ return;
+ case TCP_AO_ALGO_AES_128_CMAC: {
+ struct aes_cmac_key k;
- return 0;
-clear_hash:
- memset(key, 0, tcp_ao_digest_size(mkt));
- return 1;
+ aes_cmac_preparekey(&k, mkt->key, AES_KEYSIZE_128);
+ aes_cmac(&k, input, input_len, traffic_key);
+ return;
+ }
+ default:
+ WARN_ON_ONCE(1); /* algo was validated earlier. */
+ }
}
bool tcp_ao_ignore_icmp(const struct sock *sk, int family, int type, int code)
{
bool ignore_icmp = false;
@@ -252,33 +355,30 @@ static struct tcp_ao_key *tcp_ao_copy_key(struct sock *sk,
if (!new_key)
return NULL;
*new_key = *key;
INIT_HLIST_NODE(&new_key->node);
- tcp_sigpool_get(new_key->tcp_sigpool_id);
atomic64_set(&new_key->pkt_good, 0);
atomic64_set(&new_key->pkt_bad, 0);
return new_key;
}
static void tcp_ao_key_free_rcu(struct rcu_head *head)
{
struct tcp_ao_key *key = container_of(head, struct tcp_ao_key, rcu);
- tcp_sigpool_release(key->tcp_sigpool_id);
kfree_sensitive(key);
}
static void tcp_ao_info_free(struct tcp_ao_info *ao)
{
struct tcp_ao_key *key;
struct hlist_node *n;
hlist_for_each_entry_safe(key, n, &ao->head, node) {
hlist_del(&key->node);
- tcp_sigpool_release(key->tcp_sigpool_id);
kfree_sensitive(key);
}
kfree(ao);
static_branch_slow_dec_deferred(&tcp_ao_needed);
}
@@ -344,33 +444,26 @@ static int tcp_v4_ao_calc_key(struct tcp_ao_key *mkt, u8 *key,
struct kdf_input_block {
u8 counter;
u8 label[6];
struct tcp4_ao_context ctx;
__be16 outlen;
- } __packed * tmp;
- struct tcp_sigpool hp;
- int err;
-
- err = tcp_sigpool_start(mkt->tcp_sigpool_id, &hp);
- if (err)
- return err;
-
- tmp = hp.scratch;
- tmp->counter = 1;
- memcpy(tmp->label, "TCP-AO", 6);
- tmp->ctx.saddr = saddr;
- tmp->ctx.daddr = daddr;
- tmp->ctx.sport = sport;
- tmp->ctx.dport = dport;
- tmp->ctx.sisn = sisn;
- tmp->ctx.disn = disn;
- tmp->outlen = htons(tcp_ao_digest_size(mkt) * 8); /* in bits */
-
- err = tcp_ao_calc_traffic_key(mkt, key, tmp, sizeof(*tmp), &hp);
- tcp_sigpool_end(&hp);
-
- return err;
+ } __packed input = {
+ .counter = 1,
+ .label = "TCP-AO",
+ .ctx = {
+ .saddr = saddr,
+ .daddr = daddr,
+ .sport = sport,
+ .dport = dport,
+ .sisn = sisn,
+ .disn = disn,
+ },
+ .outlen = htons(tcp_ao_digest_size(mkt) * 8), /* in bits */
+ };
+
+ tcp_ao_calc_traffic_key(mkt, key, &input, sizeof(input));
+ return 0;
}
int tcp_v4_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
const struct sock *sk,
__be32 sisn, __be32 disn, bool send)
@@ -433,60 +526,57 @@ static int tcp_ao_calc_key_skb(struct tcp_ao_key *mkt, u8 *key,
return tcp_v6_ao_calc_key_skb(mkt, key, skb, sisn, disn);
#endif
return -EAFNOSUPPORT;
}
-static int tcp_v4_ao_hash_pseudoheader(struct tcp_sigpool *hp,
+static int tcp_v4_ao_hash_pseudoheader(struct tcp_ao_mac_ctx *mac_ctx,
__be32 daddr, __be32 saddr,
int nbytes)
{
- struct tcp4_pseudohdr *bp;
- struct scatterlist sg;
+ struct tcp4_pseudohdr phdr = {
+ .saddr = saddr,
+ .daddr = daddr,
+ .pad = 0,
+ .protocol = IPPROTO_TCP,
+ .len = cpu_to_be16(nbytes),
+ };
- bp = hp->scratch;
- bp->saddr = saddr;
- bp->daddr = daddr;
- bp->pad = 0;
- bp->protocol = IPPROTO_TCP;
- bp->len = cpu_to_be16(nbytes);
-
- sg_init_one(&sg, bp, sizeof(*bp));
- ahash_request_set_crypt(hp->req, &sg, NULL, sizeof(*bp));
- return crypto_ahash_update(hp->req);
+ tcp_ao_mac_update(mac_ctx, &phdr, sizeof(phdr));
+ return 0;
}
static int tcp_ao_hash_pseudoheader(unsigned short int family,
const struct sock *sk,
const struct sk_buff *skb,
- struct tcp_sigpool *hp, int nbytes)
+ struct tcp_ao_mac_ctx *mac_ctx, int nbytes)
{
const struct tcphdr *th = tcp_hdr(skb);
/* TODO: Can we rely on checksum being zero to mean outbound pkt? */
if (!th->check) {
if (family == AF_INET)
- return tcp_v4_ao_hash_pseudoheader(hp, sk->sk_daddr,
+ return tcp_v4_ao_hash_pseudoheader(mac_ctx, sk->sk_daddr,
sk->sk_rcv_saddr, skb->len);
#if IS_ENABLED(CONFIG_IPV6)
else if (family == AF_INET6)
- return tcp_v6_ao_hash_pseudoheader(hp, &sk->sk_v6_daddr,
+ return tcp_v6_ao_hash_pseudoheader(mac_ctx, &sk->sk_v6_daddr,
&sk->sk_v6_rcv_saddr, skb->len);
#endif
else
return -EAFNOSUPPORT;
}
if (family == AF_INET) {
const struct iphdr *iph = ip_hdr(skb);
- return tcp_v4_ao_hash_pseudoheader(hp, iph->daddr,
+ return tcp_v4_ao_hash_pseudoheader(mac_ctx, iph->daddr,
iph->saddr, skb->len);
#if IS_ENABLED(CONFIG_IPV6)
} else if (family == AF_INET6) {
const struct ipv6hdr *iph = ipv6_hdr(skb);
- return tcp_v6_ao_hash_pseudoheader(hp, &iph->daddr,
+ return tcp_v6_ao_hash_pseudoheader(mac_ctx, &iph->daddr,
&iph->saddr, skb->len);
#endif
}
return -EAFNOSUPPORT;
}
@@ -504,35 +594,24 @@ u32 tcp_ao_compute_sne(u32 next_sne, u32 next_seq, u32 seq)
}
return sne;
}
-/* tcp_ao_hash_sne(struct tcp_sigpool *hp)
- * @hp - used for hashing
- * @sne - sne value
- */
-static int tcp_ao_hash_sne(struct tcp_sigpool *hp, u32 sne)
+static void tcp_ao_hash_sne(struct tcp_ao_mac_ctx *mac_ctx, u32 sne)
{
- struct scatterlist sg;
- __be32 *bp;
-
- bp = (__be32 *)hp->scratch;
- *bp = htonl(sne);
+ __be32 sne_be32 = htonl(sne);
- sg_init_one(&sg, bp, sizeof(*bp));
- ahash_request_set_crypt(hp->req, &sg, NULL, sizeof(*bp));
- return crypto_ahash_update(hp->req);
+ tcp_ao_mac_update(mac_ctx, &sne_be32, sizeof(sne_be32));
}
-static int tcp_ao_hash_header(struct tcp_sigpool *hp,
- const struct tcphdr *th,
- bool exclude_options, u8 *hash,
- int hash_offset, int hash_len)
+static void tcp_ao_hash_header(struct tcp_ao_mac_ctx *mac_ctx,
+ const struct tcphdr *th, bool exclude_options,
+ u8 *hash, int hash_offset, int hash_len)
{
- struct scatterlist sg;
- u8 *hdr = hp->scratch;
- int err, len;
+ /* Full TCP header (th->doff << 2) should fit into scratch area. */
+ u8 hdr[60];
+ int len;
/* We are not allowed to change tcphdr, make a local copy */
if (exclude_options) {
len = sizeof(*th) + sizeof(struct tcp_ao_hdr) + hash_len;
memcpy(hdr, th, sizeof(*th));
@@ -548,126 +627,105 @@ static int tcp_ao_hash_header(struct tcp_sigpool *hp,
/* zero out tcp-ao hash */
((struct tcphdr *)hdr)->check = 0;
memset(hdr + hash_offset, 0, hash_len);
}
- sg_init_one(&sg, hdr, len);
- ahash_request_set_crypt(hp->req, &sg, NULL, len);
- err = crypto_ahash_update(hp->req);
- WARN_ON_ONCE(err != 0);
- return err;
+ tcp_ao_mac_update(mac_ctx, hdr, len);
}
int tcp_ao_hash_hdr(unsigned short int family, char *ao_hash,
struct tcp_ao_key *key, const u8 *tkey,
const union tcp_ao_addr *daddr,
const union tcp_ao_addr *saddr,
const struct tcphdr *th, u32 sne)
{
- int tkey_len = tcp_ao_digest_size(key);
int hash_offset = ao_hash - (char *)th;
- struct tcp_sigpool hp;
- void *hash_buf = NULL;
-
- hash_buf = kmalloc(tkey_len, GFP_ATOMIC);
- if (!hash_buf)
- goto clear_hash_noput;
-
- if (tcp_sigpool_start(key->tcp_sigpool_id, &hp))
- goto clear_hash_noput;
-
- if (crypto_ahash_setkey(crypto_ahash_reqtfm(hp.req), tkey, tkey_len))
- goto clear_hash;
-
- if (crypto_ahash_init(hp.req))
- goto clear_hash;
+ struct tcp_ao_mac_ctx mac_ctx;
+ u8 hash_buf[TCP_AO_MAX_MAC_LEN];
- if (tcp_ao_hash_sne(&hp, sne))
- goto clear_hash;
+ tcp_ao_mac_init(&mac_ctx, key->algo, tkey);
+ tcp_ao_hash_sne(&mac_ctx, sne);
if (family == AF_INET) {
- if (tcp_v4_ao_hash_pseudoheader(&hp, daddr->a4.s_addr,
- saddr->a4.s_addr, th->doff * 4))
- goto clear_hash;
+ tcp_v4_ao_hash_pseudoheader(&mac_ctx, daddr->a4.s_addr,
+ saddr->a4.s_addr, th->doff * 4);
#if IS_ENABLED(CONFIG_IPV6)
} else if (family == AF_INET6) {
- if (tcp_v6_ao_hash_pseudoheader(&hp, &daddr->a6,
- &saddr->a6, th->doff * 4))
- goto clear_hash;
+ tcp_v6_ao_hash_pseudoheader(&mac_ctx, &daddr->a6,
+ &saddr->a6, th->doff * 4);
#endif
} else {
WARN_ON_ONCE(1);
goto clear_hash;
}
- if (tcp_ao_hash_header(&hp, th,
- !!(key->keyflags & TCP_AO_KEYF_EXCLUDE_OPT),
- ao_hash, hash_offset, tcp_ao_maclen(key)))
- goto clear_hash;
- ahash_request_set_crypt(hp.req, NULL, hash_buf, 0);
- if (crypto_ahash_final(hp.req))
- goto clear_hash;
+ tcp_ao_hash_header(&mac_ctx, th,
+ !!(key->keyflags & TCP_AO_KEYF_EXCLUDE_OPT),
+ ao_hash, hash_offset, tcp_ao_maclen(key));
+ tcp_ao_mac_final(&mac_ctx, hash_buf);
memcpy(ao_hash, hash_buf, tcp_ao_maclen(key));
- tcp_sigpool_end(&hp);
- kfree(hash_buf);
return 0;
clear_hash:
- tcp_sigpool_end(&hp);
-clear_hash_noput:
memset(ao_hash, 0, tcp_ao_maclen(key));
- kfree(hash_buf);
return 1;
}
+static void tcp_ao_hash_skb_data(struct tcp_ao_mac_ctx *mac_ctx,
+ const struct sk_buff *skb,
+ unsigned int header_len)
+{
+ const unsigned int head_data_len = skb_headlen(skb) > header_len ?
+ skb_headlen(skb) - header_len : 0;
+ const struct skb_shared_info *shi = skb_shinfo(skb);
+ struct sk_buff *frag_iter;
+ unsigned int i;
+
+ tcp_ao_mac_update(mac_ctx, (const u8 *)tcp_hdr(skb) + header_len,
+ head_data_len);
+
+ for (i = 0; i < shi->nr_frags; ++i) {
+ const skb_frag_t *f = &shi->frags[i];
+ u32 p_off, p_len, copied;
+ const void *vaddr;
+ struct page *p;
+
+ skb_frag_foreach_page(f, skb_frag_off(f), skb_frag_size(f),
+ p, p_off, p_len, copied) {
+ vaddr = kmap_local_page(p);
+ tcp_ao_mac_update(mac_ctx, vaddr + p_off, p_len);
+ kunmap_local(vaddr);
+ }
+ }
+
+ skb_walk_frags(skb, frag_iter)
+ tcp_ao_hash_skb_data(mac_ctx, frag_iter, 0);
+}
+
int tcp_ao_hash_skb(unsigned short int family,
char *ao_hash, struct tcp_ao_key *key,
const struct sock *sk, const struct sk_buff *skb,
const u8 *tkey, int hash_offset, u32 sne)
{
const struct tcphdr *th = tcp_hdr(skb);
- int tkey_len = tcp_ao_digest_size(key);
- struct tcp_sigpool hp;
- void *hash_buf = NULL;
-
- hash_buf = kmalloc(tkey_len, GFP_ATOMIC);
- if (!hash_buf)
- goto clear_hash_noput;
-
- if (tcp_sigpool_start(key->tcp_sigpool_id, &hp))
- goto clear_hash_noput;
-
- if (crypto_ahash_setkey(crypto_ahash_reqtfm(hp.req), tkey, tkey_len))
- goto clear_hash;
-
- /* For now use sha1 by default. Depends on alg in tcp_ao_key */
- if (crypto_ahash_init(hp.req))
- goto clear_hash;
+ struct tcp_ao_mac_ctx mac_ctx;
+ u8 hash_buf[TCP_AO_MAX_MAC_LEN];
- if (tcp_ao_hash_sne(&hp, sne))
- goto clear_hash;
- if (tcp_ao_hash_pseudoheader(family, sk, skb, &hp, skb->len))
- goto clear_hash;
- if (tcp_ao_hash_header(&hp, th,
- !!(key->keyflags & TCP_AO_KEYF_EXCLUDE_OPT),
- ao_hash, hash_offset, tcp_ao_maclen(key)))
- goto clear_hash;
- if (tcp_sigpool_hash_skb_data(&hp, skb, th->doff << 2))
- goto clear_hash;
- ahash_request_set_crypt(hp.req, NULL, hash_buf, 0);
- if (crypto_ahash_final(hp.req))
+ tcp_ao_mac_init(&mac_ctx, key->algo, tkey);
+ tcp_ao_hash_sne(&mac_ctx, sne);
+ if (tcp_ao_hash_pseudoheader(family, sk, skb, &mac_ctx, skb->len))
goto clear_hash;
+ tcp_ao_hash_header(&mac_ctx, th,
+ !!(key->keyflags & TCP_AO_KEYF_EXCLUDE_OPT),
+ ao_hash, hash_offset, tcp_ao_maclen(key));
+ tcp_ao_hash_skb_data(&mac_ctx, skb, th->doff << 2);
+ tcp_ao_mac_final(&mac_ctx, hash_buf);
memcpy(ao_hash, hash_buf, tcp_ao_maclen(key));
- tcp_sigpool_end(&hp);
- kfree(hash_buf);
return 0;
clear_hash:
- tcp_sigpool_end(&hp);
-clear_hash_noput:
memset(ao_hash, 0, tcp_ao_maclen(key));
- kfree(hash_buf);
return 1;
}
int tcp_v4_ao_hash_skb(char *ao_hash, struct tcp_ao_key *key,
const struct sock *sk, const struct sk_buff *skb,
@@ -1278,11 +1336,10 @@ int tcp_ao_copy_all_matching(const struct sock *sk, struct sock *newsk,
return 0;
free_and_exit:
hlist_for_each_entry_safe(key, key_head, &new_ao->head, node) {
hlist_del(&key->node);
- tcp_sigpool_release(key->tcp_sigpool_id);
atomic_sub(tcp_ao_sizeof_key(key), &newsk->sk_omem_alloc);
kfree_sensitive(key);
}
free_ao:
kfree(new_ao);
@@ -1334,27 +1391,14 @@ static int tcp_ao_verify_ipv4(struct sock *sk, struct tcp_ao_add *cmd,
*addr = (union tcp_ao_addr *)&sin->sin_addr;
return 0;
}
-static int tcp_ao_parse_crypto(struct tcp_ao_add *cmd, struct tcp_ao_key *key)
+static int tcp_ao_parse_crypto(const struct tcp_ao_add *cmd,
+ struct tcp_ao_key *key)
{
unsigned int syn_tcp_option_space;
- bool is_kdf_aes_128_cmac = false;
- struct crypto_ahash *tfm;
- struct tcp_sigpool hp;
- void *tmp_key = NULL;
- int err;
-
- /* RFC5926, 3.1.1.2. KDF_AES_128_CMAC */
- if (!strcmp("cmac(aes128)", cmd->alg_name)) {
- strscpy(cmd->alg_name, "cmac(aes)", sizeof(cmd->alg_name));
- is_kdf_aes_128_cmac = (cmd->keylen != 16);
- tmp_key = kmalloc(cmd->keylen, GFP_KERNEL);
- if (!tmp_key)
- return -ENOMEM;
- }
key->maclen = cmd->maclen ?: 12; /* 12 is the default in RFC5925 */
/* Check: maclen + tcp-ao header <= (MAX_TCP_OPTION_SPACE - mss
* - tstamp (including sackperm)
@@ -1386,68 +1430,31 @@ static int tcp_ao_parse_crypto(struct tcp_ao_add *cmd, struct tcp_ao_key *key)
*/
syn_tcp_option_space = MAX_TCP_OPTION_SPACE;
syn_tcp_option_space -= TCPOLEN_MSS_ALIGNED;
syn_tcp_option_space -= TCPOLEN_TSTAMP_ALIGNED;
syn_tcp_option_space -= TCPOLEN_WSCALE_ALIGNED;
- if (tcp_ao_len_aligned(key) > syn_tcp_option_space) {
- err = -EMSGSIZE;
- goto err_kfree;
- }
-
- key->keylen = cmd->keylen;
- memcpy(key->key, cmd->key, cmd->keylen);
-
- err = tcp_sigpool_start(key->tcp_sigpool_id, &hp);
- if (err)
- goto err_kfree;
-
- tfm = crypto_ahash_reqtfm(hp.req);
- if (is_kdf_aes_128_cmac) {
- void *scratch = hp.scratch;
- struct scatterlist sg;
-
- memcpy(tmp_key, cmd->key, cmd->keylen);
- sg_init_one(&sg, tmp_key, cmd->keylen);
-
- /* Using zero-key of 16 bytes as described in RFC5926 */
- memset(scratch, 0, 16);
- err = crypto_ahash_setkey(tfm, scratch, 16);
- if (err)
- goto err_pool_end;
-
- err = crypto_ahash_init(hp.req);
- if (err)
- goto err_pool_end;
-
- ahash_request_set_crypt(hp.req, &sg, key->key, cmd->keylen);
- err = crypto_ahash_update(hp.req);
- if (err)
- goto err_pool_end;
-
- err |= crypto_ahash_final(hp.req);
- if (err)
- goto err_pool_end;
- key->keylen = 16;
+ if (tcp_ao_len_aligned(key) > syn_tcp_option_space)
+ return -EMSGSIZE;
+
+ if (key->algo == TCP_AO_ALGO_AES_128_CMAC &&
+ cmd->keylen != AES_KEYSIZE_128) {
+ /* RFC5926, 3.1.1.2. KDF_AES_128_CMAC */
+ static const u8 zeroes[AES_KEYSIZE_128];
+ struct aes_cmac_key extractor;
+
+ aes_cmac_preparekey(&extractor, zeroes, AES_KEYSIZE_128);
+ aes_cmac(&extractor, cmd->key, cmd->keylen, key->key);
+ key->keylen = AES_KEYSIZE_128;
+ } else {
+ memcpy(key->key, cmd->key, cmd->keylen);
+ key->keylen = cmd->keylen;
}
- err = crypto_ahash_setkey(tfm, key->key, key->keylen);
- if (err)
- goto err_pool_end;
-
- tcp_sigpool_end(&hp);
- kfree_sensitive(tmp_key);
-
if (tcp_ao_maclen(key) > key->digest_size)
return -EINVAL;
return 0;
-
-err_pool_end:
- tcp_sigpool_end(&hp);
-err_kfree:
- kfree_sensitive(tmp_key);
- return err;
}
#if IS_ENABLED(CONFIG_IPV6)
static int tcp_ao_verify_ipv6(struct sock *sk, struct tcp_ao_add *cmd,
union tcp_ao_addr **paddr,
@@ -1547,58 +1554,37 @@ static struct tcp_ao_info *getsockopt_ao_info(struct sock *sk)
#define TCP_AO_GET_KEYF_VALID (TCP_AO_KEYF_IFINDEX)
static struct tcp_ao_key *tcp_ao_key_alloc(struct sock *sk,
struct tcp_ao_add *cmd)
{
- const char *algo = cmd->alg_name;
- unsigned int digest_size;
- struct crypto_ahash *tfm;
+ const struct tcp_ao_algo *algo;
struct tcp_ao_key *key;
- struct tcp_sigpool hp;
- int err, pool_id;
size_t size;
/* Force null-termination of alg_name */
cmd->alg_name[ARRAY_SIZE(cmd->alg_name) - 1] = '\0';
- /* RFC5926, 3.1.1.2. KDF_AES_128_CMAC */
- if (!strcmp("cmac(aes128)", algo))
- algo = "cmac(aes)";
- else if (strcmp("hmac(sha1)", algo) &&
- strcmp("hmac(sha256)", algo) &&
- (strcmp("cmac(aes)", algo) || cmd->keylen != 16))
- return ERR_PTR(-ENOENT);
-
- /* Full TCP header (th->doff << 2) should fit into scratch area,
- * see tcp_ao_hash_header().
+ /*
+ * For backwards compatibility, accept "cmac(aes)" as an alias for
+ * "cmac(aes128)", provided that the key length is exactly 128 bits.
*/
- pool_id = tcp_sigpool_alloc_ahash(algo, 60);
- if (pool_id < 0)
- return ERR_PTR(pool_id);
-
- err = tcp_sigpool_start(pool_id, &hp);
- if (err)
- goto err_free_pool;
+ if (strcmp(cmd->alg_name, "cmac(aes)") == 0 &&
+ cmd->keylen == AES_KEYSIZE_128)
+ strscpy(cmd->alg_name, "cmac(aes128)");
- tfm = crypto_ahash_reqtfm(hp.req);
- digest_size = crypto_ahash_digestsize(tfm);
- tcp_sigpool_end(&hp);
+ algo = tcp_ao_find_algo(cmd->alg_name);
+ if (!algo)
+ return ERR_PTR(-ENOENT);
- size = sizeof(struct tcp_ao_key) + (digest_size << 1);
+ size = sizeof(struct tcp_ao_key) + (algo->digest_size << 1);
key = sock_kmalloc(sk, size, GFP_KERNEL);
- if (!key) {
- err = -ENOMEM;
- goto err_free_pool;
- }
+ if (!key)
+ return ERR_PTR(-ENOMEM);
- key->tcp_sigpool_id = pool_id;
- key->digest_size = digest_size;
+ key->algo = algo - tcp_ao_algos;
+ key->digest_size = algo->digest_size;
return key;
-
-err_free_pool:
- tcp_sigpool_release(pool_id);
- return ERR_PTR(err);
}
static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
sockptr_t optval, int optlen)
{
@@ -1755,11 +1741,10 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
WRITE_ONCE(ao_info->rnext_key, key);
return 0;
err_free_sock:
atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc);
- tcp_sigpool_release(key->tcp_sigpool_id);
kfree_sensitive(key);
err_free_ao:
if (first)
kfree(ao_info);
return ret;
@@ -2286,11 +2271,15 @@ static int tcp_ao_copy_mkts_to_user(const struct sock *sk,
opt_out.keylen = key->keylen;
opt_out.ifindex = key->l3index;
opt_out.pkt_good = atomic64_read(&key->pkt_good);
opt_out.pkt_bad = atomic64_read(&key->pkt_bad);
memcpy(&opt_out.key, key->key, key->keylen);
- tcp_sigpool_algo(key->tcp_sigpool_id, opt_out.alg_name, 64);
+ if (key->algo == TCP_AO_ALGO_AES_128_CMAC)
+ /* This is needed for backwards compatibility. */
+ strscpy(opt_out.alg_name, "cmac(aes)");
+ else
+ strscpy(opt_out.alg_name, tcp_ao_algos[key->algo].name);
/* Copy key to user */
if (copy_to_sockptr_offset(optval, out_offset,
&opt_out, bytes_to_write))
return -EFAULT;
diff --git a/net/ipv6/tcp_ao.c b/net/ipv6/tcp_ao.c
index 3c09ac26206e..2dcfe9dda7f4 100644
--- a/net/ipv6/tcp_ao.c
+++ b/net/ipv6/tcp_ao.c
@@ -5,11 +5,10 @@
*
* Authors: Dmitry Safonov <dima@arista.com>
* Francesco Ruggeri <fruggeri@arista.com>
* Salam Noureddine <noureddine@arista.com>
*/
-#include <crypto/hash.h>
#include <linux/tcp.h>
#include <net/tcp.h>
#include <net/ipv6.h>
@@ -22,33 +21,26 @@ static int tcp_v6_ao_calc_key(struct tcp_ao_key *mkt, u8 *key,
struct kdf_input_block {
u8 counter;
u8 label[6];
struct tcp6_ao_context ctx;
__be16 outlen;
- } __packed * tmp;
- struct tcp_sigpool hp;
- int err;
-
- err = tcp_sigpool_start(mkt->tcp_sigpool_id, &hp);
- if (err)
- return err;
-
- tmp = hp.scratch;
- tmp->counter = 1;
- memcpy(tmp->label, "TCP-AO", 6);
- tmp->ctx.saddr = *saddr;
- tmp->ctx.daddr = *daddr;
- tmp->ctx.sport = sport;
- tmp->ctx.dport = dport;
- tmp->ctx.sisn = sisn;
- tmp->ctx.disn = disn;
- tmp->outlen = htons(tcp_ao_digest_size(mkt) * 8); /* in bits */
-
- err = tcp_ao_calc_traffic_key(mkt, key, tmp, sizeof(*tmp), &hp);
- tcp_sigpool_end(&hp);
-
- return err;
+ } __packed input = {
+ .counter = 1,
+ .label = "TCP-AO",
+ .ctx = {
+ .saddr = *saddr,
+ .daddr = *daddr,
+ .sport = sport,
+ .dport = dport,
+ .sisn = sisn,
+ .disn = disn,
+ },
+ .outlen = htons(tcp_ao_digest_size(mkt) * 8), /* in bits */
+ };
+
+ tcp_ao_calc_traffic_key(mkt, key, &input, sizeof(input));
+ return 0;
}
int tcp_v6_ao_calc_key_skb(struct tcp_ao_key *mkt, u8 *key,
const struct sk_buff *skb,
__be32 sisn, __be32 disn)
@@ -110,27 +102,24 @@ struct tcp_ao_key *tcp_v6_ao_lookup_rsk(const struct sock *sk,
l3index = l3mdev_master_ifindex_by_index(sock_net(sk), ireq->ir_iif);
return tcp_ao_do_lookup(sk, l3index, (union tcp_ao_addr *)addr,
AF_INET6, sndid, rcvid);
}
-int tcp_v6_ao_hash_pseudoheader(struct tcp_sigpool *hp,
+int tcp_v6_ao_hash_pseudoheader(struct tcp_ao_mac_ctx *mac_ctx,
const struct in6_addr *daddr,
const struct in6_addr *saddr, int nbytes)
{
- struct tcp6_pseudohdr *bp;
- struct scatterlist sg;
-
- bp = hp->scratch;
/* 1. TCP pseudo-header (RFC2460) */
- bp->saddr = *saddr;
- bp->daddr = *daddr;
- bp->len = cpu_to_be32(nbytes);
- bp->protocol = cpu_to_be32(IPPROTO_TCP);
-
- sg_init_one(&sg, bp, sizeof(*bp));
- ahash_request_set_crypt(hp->req, &sg, NULL, sizeof(*bp));
- return crypto_ahash_update(hp->req);
+ struct tcp6_pseudohdr phdr = {
+ .saddr = *saddr,
+ .daddr = *daddr,
+ .len = cpu_to_be32(nbytes),
+ .protocol = cpu_to_be32(IPPROTO_TCP),
+ };
+
+ tcp_ao_mac_update(mac_ctx, &phdr, sizeof(phdr));
+ return 0;
}
int tcp_v6_ao_hash_skb(char *ao_hash, struct tcp_ao_key *key,
const struct sock *sk, const struct sk_buff *skb,
const u8 *tkey, int hash_offset, u32 sne)
diff --git a/tools/testing/selftests/net/tcp_ao/config b/tools/testing/selftests/net/tcp_ao/config
index 47228a7d0b90..1b120bfd89c4 100644
--- a/tools/testing/selftests/net/tcp_ao/config
+++ b/tools/testing/selftests/net/tcp_ao/config
@@ -1,8 +1,5 @@
-CONFIG_CRYPTO_CMAC=y
-CONFIG_CRYPTO_HMAC=y
-CONFIG_CRYPTO_SHA1=y
CONFIG_IPV6=y
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_NET_L3_MASTER_DEV=y
CONFIG_NET_VRF=y
CONFIG_TCP_AO=y
--
2.54.0
^ permalink raw reply related
* [PATCH net-next v2 3/5] net/tcp-ao: Use stack-allocated MAC and traffic_key buffers
From: Eric Biggers @ 2026-04-27 17:27 UTC (permalink / raw)
To: netdev
Cc: linux-crypto, linux-kernel, Eric Dumazet, Neal Cardwell,
Kuniyuki Iwashima, David S . Miller, David Ahern, Jakub Kicinski,
Paolo Abeni, Simon Horman, Ard Biesheuvel, Jason A . Donenfeld,
Herbert Xu, Dmitry Safonov, Eric Biggers
In-Reply-To: <20260427172727.9310-1-ebiggers@kernel.org>
Now that the maximum MAC and traffic key lengths are statically-known
small values, allocate MACs and traffic keys on the stack instead of
with kmalloc. This eliminates multiple failure-prone GFP_ATOMIC
allocations.
Note that some cases such as tcp_ao_prepare_reset() are left unchanged
for now since they would require slightly wider changes.
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
net/ipv4/tcp_ao.c | 44 +++++++++++---------------------------------
net/ipv6/tcp_ao.c | 17 +++++------------
2 files changed, 16 insertions(+), 45 deletions(-)
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index 0d24cbd66c9a..69f1d6d26562 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -737,26 +737,19 @@ int tcp_v4_ao_hash_skb(char *ao_hash, struct tcp_ao_key *key,
int tcp_v4_ao_synack_hash(char *ao_hash, struct tcp_ao_key *ao_key,
struct request_sock *req, const struct sk_buff *skb,
int hash_offset, u32 sne)
{
- void *hash_buf = NULL;
+ u8 tkey_buf[TCP_AO_MAX_TRAFFIC_KEY_LEN];
int err;
- hash_buf = kmalloc(tcp_ao_digest_size(ao_key), GFP_ATOMIC);
- if (!hash_buf)
- return -ENOMEM;
-
- err = tcp_v4_ao_calc_key_rsk(ao_key, hash_buf, req);
+ err = tcp_v4_ao_calc_key_rsk(ao_key, tkey_buf, req);
if (err)
- goto out;
+ return err;
- err = tcp_ao_hash_skb(AF_INET, ao_hash, ao_key, req_to_sk(req), skb,
- hash_buf, hash_offset, sne);
-out:
- kfree(hash_buf);
- return err;
+ return tcp_ao_hash_skb(AF_INET, ao_hash, ao_key, req_to_sk(req), skb,
+ tkey_buf, hash_offset, sne);
}
struct tcp_ao_key *tcp_v4_ao_lookup_rsk(const struct sock *sk,
struct request_sock *req,
int sndid, int rcvid)
@@ -867,13 +860,13 @@ int tcp_ao_prepare_reset(const struct sock *sk, struct sk_buff *skb,
int tcp_ao_transmit_skb(struct sock *sk, struct sk_buff *skb,
struct tcp_ao_key *key, struct tcphdr *th,
__u8 *hash_location)
{
struct tcp_skb_cb *tcb = TCP_SKB_CB(skb);
+ u8 tkey_buf[TCP_AO_MAX_TRAFFIC_KEY_LEN];
struct tcp_sock *tp = tcp_sk(sk);
struct tcp_ao_info *ao;
- void *tkey_buf = NULL;
u8 *traffic_key;
u32 sne;
ao = rcu_dereference_protected(tcp_sk(sk)->ao_info,
lockdep_sock_is_held(sk));
@@ -881,13 +874,10 @@ int tcp_ao_transmit_skb(struct sock *sk, struct sk_buff *skb,
if (unlikely(tcb->tcp_flags & TCPHDR_SYN)) {
__be32 disn;
if (!(tcb->tcp_flags & TCPHDR_ACK)) {
disn = 0;
- tkey_buf = kmalloc(tcp_ao_digest_size(key), GFP_ATOMIC);
- if (!tkey_buf)
- return -ENOMEM;
traffic_key = tkey_buf;
} else {
disn = ao->risn;
}
tp->af_specific->ao_calc_key_sk(key, traffic_key,
@@ -895,11 +885,10 @@ int tcp_ao_transmit_skb(struct sock *sk, struct sk_buff *skb,
}
sne = tcp_ao_compute_sne(READ_ONCE(ao->snd_sne), READ_ONCE(tp->snd_una),
ntohl(th->seq));
tp->af_specific->calc_ao_hash(hash_location, key, sk, skb, traffic_key,
hash_location - (u8 *)th, sne);
- kfree(tkey_buf);
return 0;
}
static struct tcp_ao_key *tcp_ao_inbound_lookup(unsigned short int family,
const struct sock *sk, const struct sk_buff *skb,
@@ -961,54 +950,48 @@ tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,
const struct tcp_ao_hdr *aoh, struct tcp_ao_key *key,
u8 *traffic_key, u8 *phash, u32 sne, int l3index)
{
const struct tcphdr *th = tcp_hdr(skb);
u8 maclen = tcp_ao_hdr_maclen(aoh);
- void *hash_buf = NULL;
+ u8 hash_buf[TCP_AO_MAX_MAC_LEN];
if (maclen != tcp_ao_maclen(key)) {
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOBAD);
atomic64_inc(&info->counters.pkt_bad);
atomic64_inc(&key->pkt_bad);
trace_tcp_ao_wrong_maclen(sk, skb, aoh->keyid,
aoh->rnext_keyid, maclen);
return SKB_DROP_REASON_TCP_AOFAILURE;
}
- hash_buf = kmalloc(tcp_ao_digest_size(key), GFP_ATOMIC);
- if (!hash_buf)
- return SKB_DROP_REASON_NOT_SPECIFIED;
-
/* XXX: make it per-AF callback? */
tcp_ao_hash_skb(family, hash_buf, key, sk, skb, traffic_key,
(phash - (u8 *)th), sne);
if (crypto_memneq(phash, hash_buf, maclen)) {
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOBAD);
atomic64_inc(&info->counters.pkt_bad);
atomic64_inc(&key->pkt_bad);
trace_tcp_ao_mismatch(sk, skb, aoh->keyid,
aoh->rnext_keyid, maclen);
- kfree(hash_buf);
return SKB_DROP_REASON_TCP_AOFAILURE;
}
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOGOOD);
atomic64_inc(&info->counters.pkt_good);
atomic64_inc(&key->pkt_good);
- kfree(hash_buf);
return SKB_NOT_DROPPED_YET;
}
enum skb_drop_reason
tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
unsigned short int family, const struct request_sock *req,
int l3index, const struct tcp_ao_hdr *aoh)
{
+ u8 tkey_buf[TCP_AO_MAX_TRAFFIC_KEY_LEN];
const struct tcphdr *th = tcp_hdr(skb);
u8 maclen = tcp_ao_hdr_maclen(aoh);
u8 *phash = (u8 *)(aoh + 1); /* hash goes just after the header */
struct tcp_ao_info *info;
- enum skb_drop_reason ret;
struct tcp_ao_key *key;
__be32 sisn, disn;
u8 *traffic_key;
int state;
u32 sne = 0;
@@ -1112,18 +1095,13 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
} else {
WARN_ONCE(1, "TCP-AO: Unexpected sk_state %d", state);
return SKB_DROP_REASON_TCP_AOFAILURE;
}
verify_hash:
- traffic_key = kmalloc(tcp_ao_digest_size(key), GFP_ATOMIC);
- if (!traffic_key)
- return SKB_DROP_REASON_NOT_SPECIFIED;
- tcp_ao_calc_key_skb(key, traffic_key, skb, sisn, disn, family);
- ret = tcp_ao_verify_hash(sk, skb, family, info, aoh, key,
- traffic_key, phash, sne, l3index);
- kfree(traffic_key);
- return ret;
+ tcp_ao_calc_key_skb(key, tkey_buf, skb, sisn, disn, family);
+ return tcp_ao_verify_hash(sk, skb, family, info, aoh, key,
+ tkey_buf, phash, sne, l3index);
key_not_found:
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOKEYNOTFOUND);
atomic64_inc(&info->counters.key_not_found);
trace_tcp_ao_key_not_found(sk, skb, aoh->keyid,
diff --git a/net/ipv6/tcp_ao.c b/net/ipv6/tcp_ao.c
index 2dcfe9dda7f4..bf30b970181d 100644
--- a/net/ipv6/tcp_ao.c
+++ b/net/ipv6/tcp_ao.c
@@ -136,22 +136,15 @@ int tcp_v6_parse_ao(struct sock *sk, int cmd,
int tcp_v6_ao_synack_hash(char *ao_hash, struct tcp_ao_key *ao_key,
struct request_sock *req, const struct sk_buff *skb,
int hash_offset, u32 sne)
{
- void *hash_buf = NULL;
+ u8 tkey_buf[TCP_AO_MAX_TRAFFIC_KEY_LEN];
int err;
- hash_buf = kmalloc(tcp_ao_digest_size(ao_key), GFP_ATOMIC);
- if (!hash_buf)
- return -ENOMEM;
-
- err = tcp_v6_ao_calc_key_rsk(ao_key, hash_buf, req);
+ err = tcp_v6_ao_calc_key_rsk(ao_key, tkey_buf, req);
if (err)
- goto out;
+ return err;
- err = tcp_ao_hash_skb(AF_INET6, ao_hash, ao_key, req_to_sk(req), skb,
- hash_buf, hash_offset, sne);
-out:
- kfree(hash_buf);
- return err;
+ return tcp_ao_hash_skb(AF_INET6, ao_hash, ao_key, req_to_sk(req), skb,
+ tkey_buf, hash_offset, sne);
}
--
2.54.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox