* [PATCH for-next 0/4] io_uring/zc fix and improvements
@ 2022-07-25 9:52 Pavel Begunkov
2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw)
To: io-uring; +Cc: Jens Axboe, asml.silence
It mainly fixes net/zc memory accounting error handling and then makes
it more consistent with registered buffers mem accounting.
Pavel Begunkov (4):
io_uring/net: improve io_get_notif_slot types
io_uring/net: checks errors of zc mem accounting
io_uring/net: make page accounting more consistent
io_uring/net: use unsigned for flags
io_uring/net.c | 8 +++++---
io_uring/notif.c | 9 ++++-----
io_uring/notif.h | 21 ++++++++++++++++++++-
io_uring/rsrc.c | 12 ++++--------
io_uring/rsrc.h | 9 +++++++++
5 files changed, 42 insertions(+), 17 deletions(-)
--
2.37.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting Pavel Begunkov ` (3 subsequent siblings) 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence Don't use signed int for slot indexing. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/notif.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/io_uring/notif.h b/io_uring/notif.h index 6cd73d7b965b..3e05d2cecb6f 100644 --- a/io_uring/notif.h +++ b/io_uring/notif.h @@ -67,7 +67,7 @@ static inline struct io_notif *io_get_notif(struct io_ring_ctx *ctx, } static inline struct io_notif_slot *io_get_notif_slot(struct io_ring_ctx *ctx, - int idx) + unsigned idx) __must_hold(&ctx->uring_lock) { if (idx >= ctx->nr_notif_slots) -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 3/4] io_uring/net: make page accounting more consistent Pavel Begunkov ` (2 subsequent siblings) 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence mm_account_pinned_pages() may fail, don't ignore the return value. Fixes: d3b8269075f67 ("io_uring: account locked pages for non-fixed zc") Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/net.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/io_uring/net.c b/io_uring/net.c index 62be89837d82..8fb8469c3315 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -985,7 +985,9 @@ int io_sendzc(struct io_kiocb *req, unsigned int issue_flags) &msg.msg_iter); if (unlikely(ret)) return ret; - mm_account_pinned_pages(¬if->uarg.mmp, zc->len); + ret = mm_account_pinned_pages(¬if->uarg.mmp, zc->len); + if (unlikely(ret)) + return ret; } if (zc->addr) { -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH for-next 3/4] io_uring/net: make page accounting more consistent 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 4/4] io_uring/net: use unsigned for flags Pavel Begunkov 2022-07-25 15:49 ` [PATCH for-next 0/4] io_uring/zc fix and improvements Jens Axboe 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence Make network page accounting more consistent with how buffer registration is working, i.e. account all memory to ctx->user. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/net.c | 2 +- io_uring/notif.c | 9 ++++----- io_uring/notif.h | 19 +++++++++++++++++++ io_uring/rsrc.c | 12 ++++-------- io_uring/rsrc.h | 9 +++++++++ 5 files changed, 37 insertions(+), 14 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index 8fb8469c3315..c13d971c7826 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -985,7 +985,7 @@ int io_sendzc(struct io_kiocb *req, unsigned int issue_flags) &msg.msg_iter); if (unlikely(ret)) return ret; - ret = mm_account_pinned_pages(¬if->uarg.mmp, zc->len); + ret = io_notif_account_mem(notif, zc->len); if (unlikely(ret)) return ret; } diff --git a/io_uring/notif.c b/io_uring/notif.c index a93887451bbb..e986a0ed958c 100644 --- a/io_uring/notif.c +++ b/io_uring/notif.c @@ -14,12 +14,10 @@ static void __io_notif_complete_tw(struct callback_head *cb) struct io_notif *notif = container_of(cb, struct io_notif, task_work); struct io_rsrc_node *rsrc_node = notif->rsrc_node; struct io_ring_ctx *ctx = notif->ctx; - struct mmpin *mmp = ¬if->uarg.mmp; - if (mmp->user) { - atomic_long_sub(mmp->num_pg, &mmp->user->locked_vm); - free_uid(mmp->user); - mmp->user = NULL; + if (notif->account_pages && ctx->user) { + __io_unaccount_mem(ctx->user, notif->account_pages); + notif->account_pages = 0; } if (likely(notif->task)) { io_put_task(notif->task, 1); @@ -121,6 +119,7 @@ struct io_notif *io_alloc_notif(struct io_ring_ctx *ctx, notif->ctx = ctx; notif->uarg.flags = SKBFL_ZEROCOPY_FRAG | SKBFL_DONT_ORPHAN; notif->uarg.callback = io_uring_tx_zerocopy_callback; + notif->account_pages = 0; } notif->seq = slot->seq++; diff --git a/io_uring/notif.h b/io_uring/notif.h index 3e05d2cecb6f..d6f366b1518b 100644 --- a/io_uring/notif.h +++ b/io_uring/notif.h @@ -5,6 +5,8 @@ #include <net/sock.h> #include <linux/nospec.h> +#include "rsrc.h" + #define IO_NOTIF_SPLICE_BATCH 32 #define IORING_MAX_NOTIF_SLOTS (1U << 10) @@ -23,6 +25,8 @@ struct io_notif { /* hook into ctx->notif_list and ctx->notif_list_locked */ struct list_head cache_node; + unsigned long account_pages; + union { struct callback_head task_work; struct work_struct commit_work; @@ -85,3 +89,18 @@ static inline void io_notif_slot_flush_submit(struct io_notif_slot *slot, } io_notif_slot_flush(slot); } + +static inline int io_notif_account_mem(struct io_notif *notif, unsigned len) +{ + struct io_ring_ctx *ctx = notif->ctx; + unsigned nr_pages = (len >> PAGE_SHIFT) + 2; + int ret; + + if (ctx->user) { + ret = __io_account_mem(ctx->user, nr_pages); + if (ret) + return ret; + notif->account_pages += nr_pages; + } + return 0; +} diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 9165fdf64269..59704b9ac537 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -44,17 +44,13 @@ void io_rsrc_refs_drop(struct io_ring_ctx *ctx) } } -static inline void __io_unaccount_mem(struct user_struct *user, - unsigned long nr_pages) -{ - atomic_long_sub(nr_pages, &user->locked_vm); -} - -static inline int __io_account_mem(struct user_struct *user, - unsigned long nr_pages) +int __io_account_mem(struct user_struct *user, unsigned long nr_pages) { unsigned long page_limit, cur_pages, new_pages; + if (!nr_pages) + return 0; + /* Don't allow more pages than we can safely lock */ page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; diff --git a/io_uring/rsrc.h b/io_uring/rsrc.h index 21813a23215f..f3a9a177941f 100644 --- a/io_uring/rsrc.h +++ b/io_uring/rsrc.h @@ -169,4 +169,13 @@ static inline u64 *io_get_tag_slot(struct io_rsrc_data *data, unsigned int idx) int io_rsrc_update(struct io_kiocb *req, unsigned int issue_flags); int io_rsrc_update_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe); + +int __io_account_mem(struct user_struct *user, unsigned long nr_pages); + +static inline void __io_unaccount_mem(struct user_struct *user, + unsigned long nr_pages) +{ + atomic_long_sub(nr_pages, &user->locked_vm); +} + #endif -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH for-next 4/4] io_uring/net: use unsigned for flags 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov ` (2 preceding siblings ...) 2022-07-25 9:52 ` [PATCH for-next 3/4] io_uring/net: make page accounting more consistent Pavel Begunkov @ 2022-07-25 9:52 ` Pavel Begunkov 2022-07-25 15:49 ` [PATCH for-next 0/4] io_uring/zc fix and improvements Jens Axboe 4 siblings, 0 replies; 6+ messages in thread From: Pavel Begunkov @ 2022-07-25 9:52 UTC (permalink / raw) To: io-uring; +Cc: Jens Axboe, asml.silence Use unsigned int type for msg flags. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- io_uring/net.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index c13d971c7826..8276b9537194 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -55,10 +55,10 @@ struct io_sr_msg { struct user_msghdr __user *umsg; void __user *buf; }; - int msg_flags; + unsigned msg_flags; + unsigned flags; size_t len; size_t done_io; - unsigned int flags; }; struct io_sendzc { -- 2.37.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH for-next 0/4] io_uring/zc fix and improvements 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov ` (3 preceding siblings ...) 2022-07-25 9:52 ` [PATCH for-next 4/4] io_uring/net: use unsigned for flags Pavel Begunkov @ 2022-07-25 15:49 ` Jens Axboe 4 siblings, 0 replies; 6+ messages in thread From: Jens Axboe @ 2022-07-25 15:49 UTC (permalink / raw) To: asml.silence, io-uring On Mon, 25 Jul 2022 10:52:02 +0100, Pavel Begunkov wrote: > It mainly fixes net/zc memory accounting error handling and then makes > it more consistent with registered buffers mem accounting. > > Pavel Begunkov (4): > io_uring/net: improve io_get_notif_slot types > io_uring/net: checks errors of zc mem accounting > io_uring/net: make page accounting more consistent > io_uring/net: use unsigned for flags > > [...] Applied, thanks! [1/4] io_uring/net: improve io_get_notif_slot types commit: cb309ae49da7a7c28f0051deea13970291134fac [2/4] io_uring/net: checks errors of zc mem accounting commit: 2e32ba5607ee2b668baa8831dd74f7cc867a1f7e [3/4] io_uring/net: make page accounting more consistent commit: 6a9ce66f4d0872861e0bbc67eee6ce5dca5dd886 [4/4] io_uring/net: use unsigned for flags commit: 293402e564a7391f38541c7694e736f5fde20aea Best regards, -- Jens Axboe ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-07-25 15:49 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-07-25 9:52 [PATCH for-next 0/4] io_uring/zc fix and improvements Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 1/4] io_uring/net: improve io_get_notif_slot types Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 2/4] io_uring/net: checks errors of zc mem accounting Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 3/4] io_uring/net: make page accounting more consistent Pavel Begunkov 2022-07-25 9:52 ` [PATCH for-next 4/4] io_uring/net: use unsigned for flags Pavel Begunkov 2022-07-25 15:49 ` [PATCH for-next 0/4] io_uring/zc fix and improvements Jens Axboe
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.