* Re: [net-next 0/3] Per epoll context busy poll support
From: Eric Dumazet @ 2024-01-24 8:20 UTC (permalink / raw)
To: Joe Damato
Cc: netdev, linux-kernel, chuck.lever, jlayton, linux-api, brauner,
davem, alexander.duyck, sridhar.samudrala, kuba, Wei Wang
In-Reply-To: <20240124025359.11419-1-jdamato@fastly.com>
On Wed, Jan 24, 2024 at 3:54 AM Joe Damato <jdamato@fastly.com> wrote:
>
> Greetings:
>
> TL;DR This builds on commit bf3b9f6372c4 ("epoll: Add busy poll support to
> epoll with socket fds.") by allowing user applications to enable
> epoll-based busy polling and set a busy poll packet budget on a per epoll
> context basis.
>
> To allow for this, two ioctls have been added for epoll contexts for
> getting and setting a new struct, struct epoll_params.
>
> This makes epoll-based busy polling much more usable for user
> applications than the current system-wide sysctl and hardcoded budget.
>
> Longer explanation:
>
> Presently epoll has support for a very useful form of busy poll based on
> the incoming NAPI ID (see also: SO_INCOMING_NAPI_ID [1]).
>
> This form of busy poll allows epoll_wait to drive NAPI packet processing
> which allows for a few interesting user application designs which can
> reduce latency and also potentially improve L2/L3 cache hit rates by
> deferring NAPI until userland has finished its work.
>
> The documentation available on this is, IMHO, a bit confusing so please
> allow me to explain how one might use this:
>
> 1. Ensure each application thread has its own epoll instance mapping
> 1-to-1 with NIC RX queues. An n-tuple filter would likely be used to
> direct connections with specific dest ports to these queues.
>
> 2. Optionally: Setup IRQ coalescing for the NIC RX queues where busy
> polling will occur. This can help avoid the userland app from being
> pre-empted by a hard IRQ while userland is running. Note this means that
> userland must take care to call epoll_wait and not take too long in
> userland since it now drives NAPI via epoll_wait.
>
> 3. Ensure that all incoming connections added to an epoll instance
> have the same NAPI ID. This can be done with a BPF filter when
> SO_REUSEPORT is used or getsockopt + SO_INCOMING_NAPI_ID when a single
> accept thread is used which dispatches incoming connections to threads.
>
> 4. Lastly, busy poll must be enabled via a sysctl
> (/proc/sys/net/core/busy_poll).
>
> The unfortunate part about step 4 above is that this enables busy poll
> system-wide which affects all user applications on the system,
> including epoll-based network applications which were not intended to
> be used this way or applications where increased CPU usage for lower
> latency network processing is unnecessary or not desirable.
>
> If the user wants to run one low latency epoll-based server application
> with epoll-based busy poll, but would like to run the rest of the
> applications on the system (which may also use epoll) without busy poll,
> this system-wide sysctl presents a significant problem.
>
> This change preserves the system-wide sysctl, but adds a mechanism (via
> ioctl) to enable or disable busy poll for epoll contexts as needed by
> individual applications, making epoll-based busy poll more usable.
>
I think this description missed the napi_defer_hard_irqs and
gro_flush_timeout settings ?
I would think that if an application really wants to make sure its
thread is the only one
eventually calling napi->poll(), we must make sure NIC interrupts stay masked.
Current implementations of busy poll always release NAPI_STATE_SCHED bit when
returning to user space.
It seems you want to make sure the application and only the
application calls the napi->poll()
at chosen times.
Some kind of contract is needed, and the presence of the hrtimer
(currently only driven from dev->@gro_flush_timeout)
would allow to do that correctly.
Whenever we 'trust' user space to perform the napi->poll shortly, we
also want to arm the hrtimer to eventually detect
the application took too long, to restart the other mechanisms (NIC irq based)
Note that we added the kthread based napi polling, and we are working
to add a busy polling feature to these kthreads.
allowing to completely mask NIC interrupts and further reduce latencies.
Thank you
> Thanks,
> Joe
>
> [1]: https://lore.kernel.org/lkml/20170324170836.15226.87178.stgit@localhost.localdomain/
>
> Joe Damato (3):
> eventpoll: support busy poll per epoll instance
> eventpoll: Add per-epoll busy poll packet budget
> eventpoll: Add epoll ioctl for epoll_params
>
> .../userspace-api/ioctl/ioctl-number.rst | 1 +
> fs/eventpoll.c | 99 ++++++++++++++++++-
> include/uapi/linux/eventpoll.h | 12 +++
> 3 files changed, 107 insertions(+), 5 deletions(-)
>
> --
> 2.25.1
>
^ permalink raw reply
* Re: [RFC PATCH 5/9] ntsync: Introduce NTSYNC_IOC_WAIT_ANY.
From: Arnd Bergmann @ 2024-01-24 7:56 UTC (permalink / raw)
To: Elizabeth Figura, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra
In-Reply-To: <20240124004028.16826-6-zfigura@codeweavers.com>
On Wed, Jan 24, 2024, at 01:40, Elizabeth Figura wrote:
> + if (args->timeout) {
> + struct timespec64 to;
> +
> + if (get_timespec64(&to, u64_to_user_ptr(args->timeout)))
> + return -EFAULT;
> + if (!timespec64_valid(&to))
> + return -EINVAL;
> +
> + timeout = timespec64_to_ns(&to);
> + }
Have you considered just passing the nanosecond value here?
Since you do not appear to write it back, that would avoid
the complexities of dealing with timespec layout differences
and indirection.
> + ids = kmalloc_array(count, sizeof(*ids), GFP_KERNEL);
> + if (!ids)
> + return -ENOMEM;
> + if (copy_from_user(ids, u64_to_user_ptr(args->objs),
> + array_size(count, sizeof(*ids)))) {
> + kfree(ids);
> + return -EFAULT;
> + }
This looks like memdup_user() would be slightly simpler.
Arnd
^ permalink raw reply
* Re: [RFC PATCH 8/9] ntsync: Introduce NTSYNC_IOC_PUT_MUTEX.
From: Arnd Bergmann @ 2024-01-24 7:42 UTC (permalink / raw)
To: Elizabeth Figura, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra
In-Reply-To: <20240124004028.16826-9-zfigura@codeweavers.com>
On Wed, Jan 24, 2024, at 01:40, Elizabeth Figura wrote:
> @@ -738,6 +803,8 @@ static long ntsync_char_ioctl(struct file *file,
> diff --git a/include/uapi/linux/ntsync.h b/include/uapi/linux/ntsync.h
> index 26d1b3d4847f..2e44e7e77776 100644
> --- a/include/uapi/linux/ntsync.h
> +++ b/include/uapi/linux/ntsync.h
> @@ -46,5 +46,7 @@ struct ntsync_wait_args {
> struct ntsync_wait_args)
> #define NTSYNC_IOC_CREATE_MUTEX _IOWR(NTSYNC_IOC_BASE, 5, \
> struct ntsync_mutex_args)
> +#define NTSYNC_IOC_PUT_MUTEX _IOWR(NTSYNC_IOC_BASE, 6, \
> + struct ntsync_mutex_args)
>
In your implementation, this argument is not written back to
user space, so I think this should be _IOW rather than than _IORW.
Again, no practical difference here.
Arnd
^ permalink raw reply
* Re: [RFC PATCH 1/9] ntsync: Introduce the ntsync driver and character device.
From: Arnd Bergmann @ 2024-01-24 7:38 UTC (permalink / raw)
To: Elizabeth Figura, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra
In-Reply-To: <20240124004028.16826-2-zfigura@codeweavers.com>
On Wed, Jan 24, 2024, at 01:40, Elizabeth Figura wrote:
> ntsync uses a misc device as the simplest and least intrusive uAPI interface.
>
> Each file description on the device represents an isolated NT instance, intended
> to correspond to a single NT virtual machine.
>
> Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
I'm looking at the ioctl interface to ensure it's well-formed.
Your patches look ok from that perspective, but there are a
few minor things I would check for consistency here:
> +
> +static const struct file_operations ntsync_fops = {
> + .owner = THIS_MODULE,
> + .open = ntsync_char_open,
> + .release = ntsync_char_release,
> + .unlocked_ioctl = ntsync_char_ioctl,
> + .compat_ioctl = ntsync_char_ioctl,
> + .llseek = no_llseek,
> +};
The .compat_ioctl pointer should point to compat_ptr_ioctl()
since the actual ioctl commands all take pointers instead
of interpreting the argument as a number.
On x86 and arm64 this won't make a difference as compat_ptr()
is a nop.
Arnd
^ permalink raw reply
* Re: [RFC PATCH 2/9] ntsync: Reserve a minor device number and ioctl range.
From: Elizabeth Figura @ 2024-01-24 3:43 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Arnd Bergmann, linux-kernel, linux-api, wine-devel,
André Almeida, Wolfram Sang, Arkadiusz Hiler, Peter Zijlstra
In-Reply-To: <2024012356-dove-duke-f7f6@gregkh>
On Tuesday, 23 January 2024 18:54:02 CST Greg Kroah-Hartman wrote:
> On Tue, Jan 23, 2024 at 06:40:21PM -0600, Elizabeth Figura wrote:
> > Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
> > ---
>
> Note, we can't take patches without any changelog text, and you don't
> want us to :)
>
> > Documentation/admin-guide/devices.txt | 3 ++-
> > Documentation/userspace-api/ioctl/ioctl-number.rst | 2 ++
> > drivers/misc/ntsync.c | 3 ++-
> > include/linux/miscdevice.h | 1 +
> > 4 files changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/devices.txt
> > b/Documentation/admin-guide/devices.txt index 94c98be1329a..041404397ee5
> > 100644
> > --- a/Documentation/admin-guide/devices.txt
> > +++ b/Documentation/admin-guide/devices.txt
> > @@ -376,8 +376,9 @@
> >
> > 240 = /dev/userio Serio driver testing device
> > 241 = /dev/vhost-vsock Host kernel driver for virtio
vsock
> > 242 = /dev/rfkill Turning off radio transmissions
(rfkill)
> >
> > + 243 = /dev/ntsync NT synchronization primitive
device
> >
> > - 243-254 Reserved for local use
> > + 244-254 Reserved for local use
>
> Why do you need a fixed minor number? Can't your userspace handle
> dynamic numbers? What systems require a static value?
I believe I added this because it's necessary for MODULE_ALIAS (and, more
broadly, because I was following the example of vaguely comparable devices
like /dev/loop-control). I suppose I could instead just remove MODULE_ALIAS
(or even remove the ability to compile ntsync as a module entirely).
It's a bit difficult to figure out what's the preferred way to organize things
like this (there not being a lot of precedent for this kind of driver) so I'd
appreciate any direction.
--Zeb
^ permalink raw reply
* Re: [RFC PATCH 3/9] ntsync: Introduce NTSYNC_IOC_CREATE_SEM and NTSYNC_IOC_DELETE.
From: Elizabeth Figura @ 2024-01-24 3:35 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Arnd Bergmann, linux-kernel, linux-api, wine-devel,
André Almeida, Wolfram Sang, Arkadiusz Hiler, Peter Zijlstra
In-Reply-To: <2024012301-dork-awry-c9ad@gregkh>
On Tuesday, 23 January 2024 19:14:17 CST Greg Kroah-Hartman wrote:
> On Tue, Jan 23, 2024 at 06:40:22PM -0600, Elizabeth Figura wrote:
> > +static int ntsync_create_sem(struct ntsync_device *dev, void __user
> > *argp)
> > +{
> > + struct ntsync_sem_args __user *user_args = argp;
> > + struct ntsync_sem_args args;
> > + struct ntsync_obj *sem;
> > + __u32 id;
> > + int ret;
> > +
> > + if (copy_from_user(&args, argp, sizeof(args)))
> > + return -EFAULT;
> > +
> > + if (args.count > args.max)
> > + return -EINVAL;
>
> No bounds checking on count or max?
>
> What's the relationship between count and max?
Indeed, no bounds checking. The counter is just the semaphore's internal value
and has no meaning other than that.
It's basically like an EFD_SEMAPHORE, except that the maximum is configurable
rather than always being 2**64-2.
> Some sort of real
> documentation is needed here, the changelog needs to explain this. Or
> somewhere, but as-is, this patch series is pretty unreviewable as I
> can't figure out how to review it because I don't know what it wants to
> do.
There is some comprehensive documentation in the series, but for ease of
review I will try to write a basic description of the API in each relevant
patch in v2.
^ permalink raw reply
* [net-next 3/3] eventpoll: Add epoll ioctl for epoll_params
From: Joe Damato @ 2024-01-24 2:53 UTC (permalink / raw)
To: netdev, linux-kernel
Cc: chuck.lever, jlayton, linux-api, brauner, edumazet, davem,
alexander.duyck, sridhar.samudrala, kuba, Joe Damato
In-Reply-To: <20240124025359.11419-1-jdamato@fastly.com>
Add an ioctl for getting and setting epoll_params. User programs can use
this ioctl to get and set the busy poll usec time or packet budget
params for a specific epoll context.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
.../userspace-api/ioctl/ioctl-number.rst | 1 +
fs/eventpoll.c | 41 +++++++++++++++++++
include/uapi/linux/eventpoll.h | 12 ++++++
3 files changed, 54 insertions(+)
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 457e16f06e04..b33918232f78 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -309,6 +309,7 @@ Code Seq# Include File Comments
0x89 0B-DF linux/sockios.h
0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range
0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range
+0x8A 00-1F linux/eventpoll.h
0x8B all linux/wireless.h
0x8C 00-3F WiNRADiO driver
<http://www.winradio.com.au/>
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 40bd97477b91..d973147c015c 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -869,6 +869,45 @@ static void ep_clear_and_put(struct eventpoll *ep)
ep_free(ep);
}
+static long ep_eventpoll_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ int ret;
+ struct eventpoll *ep;
+ struct epoll_params epoll_params;
+ void __user *uarg = (void __user *) arg;
+
+ if (!is_file_epoll(file))
+ return -EINVAL;
+
+ ep = file->private_data;
+
+ switch (cmd) {
+#ifdef CONFIG_NET_RX_BUSY_POLL
+ case EPIOCSPARAMS:
+ if (copy_from_user(&epoll_params, uarg, sizeof(epoll_params)))
+ return -EFAULT;
+
+ ep->busy_poll_usecs = epoll_params.busy_poll_usecs;
+ ep->busy_poll_budget = epoll_params.busy_poll_budget;
+ return 0;
+
+ case EPIOCGPARAMS:
+ memset(&epoll_params, 0, sizeof(epoll_params));
+ epoll_params.busy_poll_usecs = ep->busy_poll_usecs;
+ epoll_params.busy_poll_budget = ep->busy_poll_budget;
+ if (copy_to_user(uarg, &epoll_params, sizeof(epoll_params)))
+ return -EFAULT;
+
+ return 0;
+#endif
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
static int ep_eventpoll_release(struct inode *inode, struct file *file)
{
struct eventpoll *ep = file->private_data;
@@ -975,6 +1014,8 @@ static const struct file_operations eventpoll_fops = {
.release = ep_eventpoll_release,
.poll = ep_eventpoll_poll,
.llseek = noop_llseek,
+ .unlocked_ioctl = ep_eventpoll_ioctl,
+ .compat_ioctl = compat_ptr_ioctl,
};
/*
diff --git a/include/uapi/linux/eventpoll.h b/include/uapi/linux/eventpoll.h
index cfbcc4cc49ac..9211103779c4 100644
--- a/include/uapi/linux/eventpoll.h
+++ b/include/uapi/linux/eventpoll.h
@@ -85,4 +85,16 @@ struct epoll_event {
__u64 data;
} EPOLL_PACKED;
+struct epoll_params {
+ u64 busy_poll_usecs;
+ u16 busy_poll_budget;
+
+ /* for future fields */
+ uint8_t data[118];
+} EPOLL_PACKED;
+
+#define EPOLL_IOC_TYPE 0x8A
+#define EPIOCSPARAMS _IOW(EPOLL_IOC_TYPE, 0x01, struct epoll_params)
+#define EPIOCGPARAMS _IOR(EPOLL_IOC_TYPE, 0x02, struct epoll_params)
+
#endif /* _UAPI_LINUX_EVENTPOLL_H */
--
2.25.1
^ permalink raw reply related
* [net-next 2/3] eventpoll: Add per-epoll busy poll packet budget
From: Joe Damato @ 2024-01-24 2:53 UTC (permalink / raw)
To: netdev, linux-kernel
Cc: chuck.lever, jlayton, linux-api, brauner, edumazet, davem,
alexander.duyck, sridhar.samudrala, kuba, Joe Damato
In-Reply-To: <20240124025359.11419-1-jdamato@fastly.com>
When using epoll-based busy poll, the packet budget is hardcoded to
BUSY_POLL_BUDGET (8).
Add support for a per-epoll context busy poll packet budget. If not
specified, the default value (BUSY_POLL_BUDGET) is used.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
fs/eventpoll.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 4503fec01278..40bd97477b91 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -229,6 +229,8 @@ struct eventpoll {
unsigned int napi_id;
/* busy poll timeout */
u64 busy_poll_usecs;
+ /* busy poll packet budget */
+ u16 busy_poll_budget;
#endif
#ifdef CONFIG_DEBUG_LOCK_ALLOC
@@ -437,10 +439,14 @@ static bool ep_busy_loop_end(void *p, unsigned long start_time)
static bool ep_busy_loop(struct eventpoll *ep, int nonblock)
{
unsigned int napi_id = READ_ONCE(ep->napi_id);
+ u16 budget = READ_ONCE(ep->busy_poll_budget);
+
+ if (!budget)
+ budget = BUSY_POLL_BUDGET;
if ((napi_id >= MIN_NAPI_ID) && ep_busy_loop_on(ep)) {
napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false,
- BUSY_POLL_BUDGET);
+ budget);
if (ep_events_available(ep))
return true;
/*
@@ -2098,6 +2104,7 @@ static int do_epoll_create(int flags)
}
#ifdef CONFIG_NET_RX_BUSY_POLL
ep->busy_poll_usecs = 0;
+ ep->busy_poll_budget = 0;
#endif
ep->file = file;
fd_install(fd, file);
--
2.25.1
^ permalink raw reply related
* [net-next 1/3] eventpoll: support busy poll per epoll instance
From: Joe Damato @ 2024-01-24 2:53 UTC (permalink / raw)
To: netdev, linux-kernel
Cc: chuck.lever, jlayton, linux-api, brauner, edumazet, davem,
alexander.duyck, sridhar.samudrala, kuba, Joe Damato
In-Reply-To: <20240124025359.11419-1-jdamato@fastly.com>
Allow busy polling on a per-epoll context basis. The per-epoll context
usec timeout value is preferred, but the pre-existing system wide sysctl
value is still supported if it specified.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
fs/eventpoll.c | 49 +++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 45 insertions(+), 4 deletions(-)
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 3534d36a1474..4503fec01278 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -227,6 +227,8 @@ struct eventpoll {
#ifdef CONFIG_NET_RX_BUSY_POLL
/* used to track busy poll napi_id */
unsigned int napi_id;
+ /* busy poll timeout */
+ u64 busy_poll_usecs;
#endif
#ifdef CONFIG_DEBUG_LOCK_ALLOC
@@ -386,12 +388,44 @@ static inline int ep_events_available(struct eventpoll *ep)
READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR;
}
+/**
+ * busy_loop_ep_timeout - check if busy poll has timed out. The timeout value
+ * from the epoll instance ep is preferred, but if it is not set fallback to
+ * the system-wide global via busy_loop_timeout.
+ *
+ * @start_time: The start time used to compute the remaining time until timeout.
+ * @ep: Pointer to the eventpoll context.
+ *
+ * Return: true if the timeout has expired, false otherwise.
+ */
+static inline bool busy_loop_ep_timeout(unsigned long start_time, struct eventpoll *ep)
+{
+#ifdef CONFIG_NET_RX_BUSY_POLL
+ unsigned long bp_usec = READ_ONCE(ep->busy_poll_usecs);
+
+ if (bp_usec) {
+ unsigned long end_time = start_time + bp_usec;
+ unsigned long now = busy_loop_current_time();
+
+ return time_after(now, end_time);
+ } else {
+ return busy_loop_timeout(start_time);
+ }
+#endif
+ return true;
+}
+
#ifdef CONFIG_NET_RX_BUSY_POLL
+static bool ep_busy_loop_on(struct eventpoll *ep)
+{
+ return !!ep->busy_poll_usecs ^ net_busy_loop_on();
+}
+
static bool ep_busy_loop_end(void *p, unsigned long start_time)
{
struct eventpoll *ep = p;
- return ep_events_available(ep) || busy_loop_timeout(start_time);
+ return ep_events_available(ep) || busy_loop_ep_timeout(start_time, ep);
}
/*
@@ -404,7 +438,7 @@ static bool ep_busy_loop(struct eventpoll *ep, int nonblock)
{
unsigned int napi_id = READ_ONCE(ep->napi_id);
- if ((napi_id >= MIN_NAPI_ID) && net_busy_loop_on()) {
+ if ((napi_id >= MIN_NAPI_ID) && ep_busy_loop_on(ep)) {
napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false,
BUSY_POLL_BUDGET);
if (ep_events_available(ep))
@@ -430,7 +464,8 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi)
struct socket *sock;
struct sock *sk;
- if (!net_busy_loop_on())
+ ep = epi->ep;
+ if (!ep_busy_loop_on(ep))
return;
sock = sock_from_file(epi->ffd.file);
@@ -442,7 +477,6 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi)
return;
napi_id = READ_ONCE(sk->sk_napi_id);
- ep = epi->ep;
/* Non-NAPI IDs can be rejected
* or
@@ -466,6 +500,10 @@ static inline void ep_set_busy_poll_napi_id(struct epitem *epi)
{
}
+static inline bool ep_busy_loop_on(struct eventpoll *ep)
+{
+ return false;
+}
#endif /* CONFIG_NET_RX_BUSY_POLL */
/*
@@ -2058,6 +2096,9 @@ static int do_epoll_create(int flags)
error = PTR_ERR(file);
goto out_free_fd;
}
+#ifdef CONFIG_NET_RX_BUSY_POLL
+ ep->busy_poll_usecs = 0;
+#endif
ep->file = file;
fd_install(fd, file);
return fd;
--
2.25.1
^ permalink raw reply related
* [net-next 0/3] Per epoll context busy poll support
From: Joe Damato @ 2024-01-24 2:53 UTC (permalink / raw)
To: netdev, linux-kernel
Cc: chuck.lever, jlayton, linux-api, brauner, edumazet, davem,
alexander.duyck, sridhar.samudrala, kuba, Joe Damato
Greetings:
TL;DR This builds on commit bf3b9f6372c4 ("epoll: Add busy poll support to
epoll with socket fds.") by allowing user applications to enable
epoll-based busy polling and set a busy poll packet budget on a per epoll
context basis.
To allow for this, two ioctls have been added for epoll contexts for
getting and setting a new struct, struct epoll_params.
This makes epoll-based busy polling much more usable for user
applications than the current system-wide sysctl and hardcoded budget.
Longer explanation:
Presently epoll has support for a very useful form of busy poll based on
the incoming NAPI ID (see also: SO_INCOMING_NAPI_ID [1]).
This form of busy poll allows epoll_wait to drive NAPI packet processing
which allows for a few interesting user application designs which can
reduce latency and also potentially improve L2/L3 cache hit rates by
deferring NAPI until userland has finished its work.
The documentation available on this is, IMHO, a bit confusing so please
allow me to explain how one might use this:
1. Ensure each application thread has its own epoll instance mapping
1-to-1 with NIC RX queues. An n-tuple filter would likely be used to
direct connections with specific dest ports to these queues.
2. Optionally: Setup IRQ coalescing for the NIC RX queues where busy
polling will occur. This can help avoid the userland app from being
pre-empted by a hard IRQ while userland is running. Note this means that
userland must take care to call epoll_wait and not take too long in
userland since it now drives NAPI via epoll_wait.
3. Ensure that all incoming connections added to an epoll instance
have the same NAPI ID. This can be done with a BPF filter when
SO_REUSEPORT is used or getsockopt + SO_INCOMING_NAPI_ID when a single
accept thread is used which dispatches incoming connections to threads.
4. Lastly, busy poll must be enabled via a sysctl
(/proc/sys/net/core/busy_poll).
The unfortunate part about step 4 above is that this enables busy poll
system-wide which affects all user applications on the system,
including epoll-based network applications which were not intended to
be used this way or applications where increased CPU usage for lower
latency network processing is unnecessary or not desirable.
If the user wants to run one low latency epoll-based server application
with epoll-based busy poll, but would like to run the rest of the
applications on the system (which may also use epoll) without busy poll,
this system-wide sysctl presents a significant problem.
This change preserves the system-wide sysctl, but adds a mechanism (via
ioctl) to enable or disable busy poll for epoll contexts as needed by
individual applications, making epoll-based busy poll more usable.
Thanks,
Joe
[1]: https://lore.kernel.org/lkml/20170324170836.15226.87178.stgit@localhost.localdomain/
Joe Damato (3):
eventpoll: support busy poll per epoll instance
eventpoll: Add per-epoll busy poll packet budget
eventpoll: Add epoll ioctl for epoll_params
.../userspace-api/ioctl/ioctl-number.rst | 1 +
fs/eventpoll.c | 99 ++++++++++++++++++-
include/uapi/linux/eventpoll.h | 12 +++
3 files changed, 107 insertions(+), 5 deletions(-)
--
2.25.1
^ permalink raw reply
* [PATCH] selftests/landlock:Fix fs_test build issues with old libc
From: Hu Yadi @ 2024-01-24 2:29 UTC (permalink / raw)
To: jmorris, serge, shuah, mathieu.desnoyers, mic, amir73il, brauner,
avagin
Cc: linux-api, linux-kernel, linux-security-module, linux-kselftest,
514118380, konstantin.meskhidze, Hu.Yadi
From: "Hu.Yadi" <hu.yadi@h3c.com>
Fixes: 04f9070e99a4 ("selftests/landlock: Add tests for pseudo filesystems")
one issues comes up while building selftest/landlock on my side
(gcc 7.3/glibc-2.28/kernel-4.19)
gcc -Wall -O2 -isystem fs_test.c -lcap -o selftests/landlock/fs_test
fs_test.c:4575:9: error: initializer element is not constant
.mnt = mnt_tmp,
^~~~~~~
Signed-off-by: Hu.Yadi <hu.yadi@h3c.com>
Suggested-by: Jiao <jiaoxupo@h3c.com>
Reviewed-by: Berlin <berlin@h3c.com>
---
tools/testing/selftests/landlock/fs_test.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/landlock/fs_test.c b/tools/testing/selftests/landlock/fs_test.c
index 18e1f86a6234..1f2584b4dfce 100644
--- a/tools/testing/selftests/landlock/fs_test.c
+++ b/tools/testing/selftests/landlock/fs_test.c
@@ -40,6 +40,7 @@ int renameat2(int olddirfd, const char *oldpath, int newdirfd,
#define TMP_DIR "tmp"
#define BINARY_PATH "./true"
+#define MNT_TMP_DATA "size=4m,mode=700"
/* Paths (sibling number and depth) */
static const char dir_s1d1[] = TMP_DIR "/s1d1";
@@ -4572,7 +4573,10 @@ FIXTURE_VARIANT(layout3_fs)
/* clang-format off */
FIXTURE_VARIANT_ADD(layout3_fs, tmpfs) {
/* clang-format on */
- .mnt = mnt_tmp,
+ .mnt = {
+ .type = "tmpfs",
+ .data = MNT_TMP_DATA,
+ },
.file_path = file1_s1d1,
};
--
2.23.0
^ permalink raw reply related
* 回复: 回复: [PATCH v4] selftests/landlock:Fix two build issues
From: Huyadi @ 2024-01-24 1:54 UTC (permalink / raw)
To: 'Mickaël Salaün'
Cc: jmorris@namei.org, serge@hallyn.com, shuah@kernel.org,
mathieu.desnoyers@efficios.com, amir73il@gmail.com,
brauner@kernel.org, avagin@google.com, linux-api@vger.kernel.org,
linux-kernel@vger.kernel.org,
linux-security-module@vger.kernel.org,
linux-kselftest@vger.kernel.org, 514118380@qq.com,
konstantin.meskhidze@huawei.com
In-Reply-To: <20240123.deeT9hegh4vo@digikod.net>
>On Tue, Jan 23, 2024 at 12:04:17PM +0000, Huyadi wrote:
>>
>> >> Changes v3 -> v2:
>> >> - add helper of gettid instead of __NR_gettid
>> >> - add gcc/glibc version info in comments Changes v1 -> v2:
>> >> - fix whitespace error
>> >> - replace SYS_gettid with _NR_gettid
>> >>
>> >> tools/testing/selftests/landlock/fs_test.c | 5 ++++-
>> >> tools/testing/selftests/landlock/net_test.c | 7 ++++++-
>> >> 2 files changed, 10 insertions(+), 2 deletions(-)
>> >>
>> >> diff --git a/tools/testing/selftests/landlock/fs_test.c
>> >> b/tools/testing/selftests/landlock/fs_test.c
>> >> index 18e1f86a6234..a992cf7c0ad1 100644
>> >> --- a/tools/testing/selftests/landlock/fs_test.c
>> >> +++ b/tools/testing/selftests/landlock/fs_test.c
>> >> @@ -4572,7 +4572,10 @@ FIXTURE_VARIANT(layout3_fs)
>> >> /* clang-format off */
>> >> FIXTURE_VARIANT_ADD(layout3_fs, tmpfs) {
>> >> /* clang-format on */
>> >> - .mnt = mnt_tmp,
>> >> + .mnt = {
>> >> + .type = "tmpfs",
>> >> + .data = "size=4m,mode=700",
>> >> + },
>> >
>> >I requested some changes here.
>> >
>>
>> Could you give me some inspiration how to fix it?
>> it looks fine to me to assign value as above, which consistent with other pseudo FS tests.
>> Thanks in advance.
>
>Just add and use this for the two tmpfs data:
>#define MNT_TMP_DATA "size=4m,mode=700"
>
>You can also make the mnt_tmp variable static const.
static const is not useful for the case, the errot still, and I'll use macro definition to solve it.
thanks your warmly instruction,I'll send next version , please help to review it.
>
> >> .file_path = file1_s1d1,
> >> };
> >>
>
^ permalink raw reply
* Re: [PATCH v2 3/3] mm/mempolicy: introduce MPOL_WEIGHTED_INTERLEAVE for weighted interleaving
From: Huang, Ying @ 2024-01-24 1:51 UTC (permalink / raw)
To: Gregory Price
Cc: Gregory Price, linux-mm, linux-kernel, linux-doc, linux-fsdevel,
linux-api, corbet, akpm, honggyu.kim, rakie.kim, hyeongtak.ji,
mhocko, vtavarespetr, jgroves, ravis.opensrc, sthanneeru,
emirakhur, Hasan.Maruf, seungjun.ha, hannes, dan.j.williams,
Srinivasulu Thanneeru
In-Reply-To: <ZbAvR+U+tyLvsh8R@memverge.com>
Gregory Price <gregory.price@memverge.com> writes:
> On Tue, Jan 23, 2024 at 04:35:19PM +0800, Huang, Ying wrote:
>> Gregory Price <gregory.price@memverge.com> writes:
>>
>> > On Mon, Jan 22, 2024 at 11:54:34PM -0500, Gregory Price wrote:
>> >> >
>> >> > Can the above code be simplified as something like below?
>> >> >
>> >> > resume_node = prev_node;
>> > --- resume_weight = 0;
>> > +++ resume_weight = weights[node];
>> >> > for (...) {
>> >> > ...
>> >> > }
>> >> >
>> >>
>> >> I'll take another look at it, but this logic is annoying because of the
>> >> corner case: me->il_prev can be NUMA_NO_NODE or an actual numa node.
>> >>
>> >
>> > After a quick look, as long as no one objects to (me->il_prev) remaining
>> > NUMA_NO_NODE
>>
>> MAX_NUMNODES-1 ?
>>
>
> When setting a new policy, the il_prev gets set to NUMA_NO_NODE. It's
IIUC, it is set to MAX_NUMNODES-1 as below,
@@ -846,7 +858,8 @@ static long do_set_mempolicy(unsigned short mode, unsigned short flags,
old = current->mempolicy;
current->mempolicy = new;
- if (new && new->mode == MPOL_INTERLEAVE)
+ if (new && (new->mode == MPOL_INTERLEAVE ||
+ new->mode == MPOL_WEIGHTED_INTERLEAVE))
current->il_prev = MAX_NUMNODES-1;
task_unlock(current);
mpol_put(old);
I don't think we need to change this.
> not harmful and is just (-1), which is functionally the same as
> (MAX_NUMNODES-1) for the purpose of iterating the nodemask with
> next_node_in(). So it's fine to set (resume_node = me->il_prev)
> as discussed.
>
> I have a cleaned up function I'll push when i fix up a few other spots.
>
>> > while having a weight assigned to pol->wil.cur_weight,
>>
>> I think that it is OK.
>>
>> And, IIUC, pol->wil.cur_weight can be 0, as in
>> weighted_interleave_nodes(), if it's 0, it will be assigned to default
>> weight for the node.
>>
>
> cur_weight is different than the global weights. cur_weight tells us
> how many pages are remaining to allocate for the current node.
>
> (cur_weight = 0) can happen in two scenarios:
> - initial setting of mempolicy (NUMA_NO_NODE w/ cur_weight=0)
> - weighted_interleave_nodes decrements it down to 0
>
> Now that i'm looking at it - the second condition should not exist, and
> we can eliminate it. The logic in weighted_interleave_nodes is actually
> annoyingly unclear at the moment, so I'm going to re-factor it a bit to
> be more explicit.
I am OK with either way. Just a reminder, the first condition may be
true in alloc_pages_bulk_array_weighted_interleave() and perhaps some
other places.
--
Best Regards,
Huang, Ying
^ permalink raw reply
* Re: [RFC PATCH 0/9] NT synchronization primitive driver
From: Elizabeth Figura @ 2024-01-24 1:37 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Arnd Bergmann, linux-kernel, linux-api, wine-devel,
André Almeida, Wolfram Sang, Arkadiusz Hiler, Peter Zijlstra
In-Reply-To: <2024012319-aptly-calculate-0f88@gregkh>
On Tuesday, 23 January 2024 18:59:35 CST Greg Kroah-Hartman wrote:
> On Tue, Jan 23, 2024 at 06:40:19PM -0600, Elizabeth Figura wrote:
> > == Patches ==
> >
> > This is the first part of a 32-patch series. The series comprises 17 patches
> > which contain the actual implementation, 13 which provide self-tests, 1 to
> > update the MAINTAINERS file, and 1 to add API documentation.
>
> 32 patches? I only see 9 here, why not submit them all?
Because Documentation/process/submitting-patches.rst makes a point of asking people not to submit large patch series (and it matches the expectation of other projects I've worked with—that patches would be submitted and reviewed a few at a time). I suppose I've misunderstood that advice, though.
I'll resend with the entire series. Sorry for the noise.
^ permalink raw reply
* Re: [PATCH v3 1/3] pidfd: allow pidfd_open() on non-thread-group leaders
From: Oleg Nesterov @ 2024-01-24 1:25 UTC (permalink / raw)
To: Tycho Andersen; +Cc: Christian Brauner, linux-kernel, linux-api, Tycho Andersen
In-Reply-To: <20240123222231.GA25162@redhat.com>
On 01/23, Oleg Nesterov wrote:
>
> But to be honest I can't understand test_non_tgl_poll_exit() at all. I don't
> even understand why the process/thread created by fork_task_with_thread()
> should ever exit. And why it creates the "writer" child... Never mind, too
> late for me to read the code.
Ah, OK, it passes thread_wait_exit to fork_task_with_thread(), and this "fn"
reads sk_pair and does exit() which is actually exit_group().
Oleg.
^ permalink raw reply
* Re: [RFC PATCH 3/9] ntsync: Introduce NTSYNC_IOC_CREATE_SEM and NTSYNC_IOC_DELETE.
From: Greg Kroah-Hartman @ 2024-01-24 1:14 UTC (permalink / raw)
To: Elizabeth Figura
Cc: Arnd Bergmann, linux-kernel, linux-api, wine-devel,
André Almeida, Wolfram Sang, Arkadiusz Hiler, Peter Zijlstra
In-Reply-To: <20240124004028.16826-4-zfigura@codeweavers.com>
On Tue, Jan 23, 2024 at 06:40:22PM -0600, Elizabeth Figura wrote:
> +static int ntsync_create_sem(struct ntsync_device *dev, void __user *argp)
> +{
> + struct ntsync_sem_args __user *user_args = argp;
> + struct ntsync_sem_args args;
> + struct ntsync_obj *sem;
> + __u32 id;
> + int ret;
> +
> + if (copy_from_user(&args, argp, sizeof(args)))
> + return -EFAULT;
> +
> + if (args.count > args.max)
> + return -EINVAL;
No bounds checking on count or max?
What's the relationship between count and max? Some sort of real
documentation is needed here, the changelog needs to explain this. Or
somewhere, but as-is, this patch series is pretty unreviewable as I
can't figure out how to review it because I don't know what it wants to
do.
thanks,
greg k-h
^ permalink raw reply
* Re: [RFC PATCH 0/9] NT synchronization primitive driver
From: Greg Kroah-Hartman @ 2024-01-24 0:59 UTC (permalink / raw)
To: Elizabeth Figura
Cc: Arnd Bergmann, linux-kernel, linux-api, wine-devel,
André Almeida, Wolfram Sang, Arkadiusz Hiler, Peter Zijlstra
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
On Tue, Jan 23, 2024 at 06:40:19PM -0600, Elizabeth Figura wrote:
> == Patches ==
>
> This is the first part of a 32-patch series. The series comprises 17 patches
> which contain the actual implementation, 13 which provide self-tests, 1 to
> update the MAINTAINERS file, and 1 to add API documentation.
32 patches? I only see 9 here, why not submit them all?
> The intended semantics of the patches are broadly intended to match those of the
> corresponding Windows functions. Since I do not expect familiarity with Windows
> syscalls, however, and especially not with some of the more subtle or
> unspecified behaviour that they provide, the documentation patch included in the
> series also describes the intended behaviour in detail, and can be used as a
> specification for the rest of the series.
>
> The entire series can be retrieved or browsed here:
>
> https://repo.or.cz/linux/zf.git/shortlog/refs/heads/ntsync4
No one is going to dig elsewhere for kernel changes, sorry. Please
submit them in email for review, that's the only way we can look at them
and comment.
thanks,
greg k-h
^ permalink raw reply
* [RFC PATCH 0/9] NT synchronization primitive driver
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
This patch series introduces a new char misc driver, /dev/ntsync, which is used
to implement Windows NT synchronization primitives.
== Background ==
The Wine project emulates the Windows API in user space. One particular part of
that API, namely the NT synchronization primitives, have historically been
implemented via RPC to a dedicated "kernel" process. However, more recent
applications use these APIs more strenuously, and the overhead of RPC has become
a bottleneck.
The NT synchronization APIs are too complex to implement on top of existing
primitives without sacrificing correctness. Certain operations, such as
NtPulseEvent() or the "wait-for-all" mode of NtWaitForMultipleObjects(), require
direct control over the underlying wait queue, and implementing a wait queue
sufficiently robust for Wine in user space is not possible. This proposed
driver, therefore, implements the problematic interfaces directly in the Linux
kernel.
This driver was presented at Linux Plumbers Conference 2023. For those further
interested in the history of synchronization in Wine and past attempts to solve
this problem in user space, a recording of the presentation can be viewed here:
https://www.youtube.com/watch?v=NjU4nyWyhU8
== Performance ==
The gain in performance varies wildly depending on the application in question
and the user's hardware. For some games NT synchronization is not a bottleneck
and no change can be observed, but for others frame rate improvements of 50 to
150 percent are not atypical. The following table lists frame rate measurements
from a variety of games on a variety of hardware, taken by users Dmitry
Skvortsov, FuzzyQuills, OnMars, and myself:
Game Upstream ntsync improvement
===========================================================================
Anger Foot 69 99 43%
Call of Juarez 99.8 224.1 125%
Dirt 3 110.6 860.7 678%
Forza Horizon 5 108 160 48%
Lara Croft: Temple of Osiris 141 326 131%
Metro 2033 164.4 199.2 21%
Resident Evil 2 26 77 196%
The Crew 26 51 96%
Tiny Tina's Wonderlands 130 360 177%
Total War Saga: Troy 109 146 34%
===========================================================================
== Patches ==
This is the first part of a 32-patch series. The series comprises 17 patches
which contain the actual implementation, 13 which provide self-tests, 1 to
update the MAINTAINERS file, and 1 to add API documentation.
The intended semantics of the patches are broadly intended to match those of the
corresponding Windows functions. Since I do not expect familiarity with Windows
syscalls, however, and especially not with some of the more subtle or
unspecified behaviour that they provide, the documentation patch included in the
series also describes the intended behaviour in detail, and can be used as a
specification for the rest of the series.
The entire series can be retrieved or browsed here:
https://repo.or.cz/linux/zf.git/shortlog/refs/heads/ntsync4
The patches making use of this driver in Wine can be retrieved or browsed here:
https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync4
== Implementation ==
Some aspects of the implementation may deserve particular comment:
* In the interest of performance, each object is governed only by a single
spinlock. However, NTSYNC_IOC_WAIT_ALL requires that the state of multiple
objects be changed as a single atomic operation. In order to achieve this, we
first take a device-wide lock ("wait_all_lock") any time we are going to lock
more than one object at a time.
The maximum number of objects that can be used in a vectored wait, and
therefore the maximum that can be locked simultaneously, is 64. This number is
NT's own limit.
The acquisition of multiple spinlocks will degrade performance. This is a
conscious choice, however. Wait-for-all is known to be a very rare operation
in practice, especially with counts that approach the maximum, and it is the
intent of the ntsync driver to optimize the wait-for-any pattern at the
expense of the wait-for-all pattern as much as possible.
* NT mutexes are tied to their threads on an OS level, and the kernel includes
builtin support for "robust" mutexes. In order to keep the ntsync driver
self-contained and avoid touching more code than necessary, it does not hook
into task exit nor use pids.
Instead, the user space emulator is expected to manage thread IDs and pass
them as an argument to any relevant functions; this is the "owner" field of
ntsync_wait_args and ntsync_mutex_args.
When the emulator detects that a thread dies, it should therefore call
NTSYNC_IOC_KILL_OWNER, which will mark mutexes owned by that thread (if any)
as abandoned.
* This implementation uses a misc device mostly because it seemed like the
simplest and least obtrusive option.
Besides simplicitly of implementation, the only particularly interesting
advantage is the ability to create an arbitrary number of "contexts"
(corresponding to Windows virtual machines) which are self-contained and
shareable across multiple processes; this maps nicely to file descriptions
(i.e. struct file). This is not impossible with syscalls of course but would
require an extra argument.
On the other hand, there is no reason to forbid using ntsync by default from
user-mode processes, and (as far as I understand) to do so with a char device
requires explicit configuration by e.g. udev or init. Since this is done with
e.g. fuse, I assume this is the model to follow, but I may have chosen
something deprecated.
* ntsync is module-capable mostly because there was nothing preventing it, and
because it aided development. I am not aware of any reason why being a module
is required, though.
* The misc minor number has not been reserved with LANANA. I am not sure at what
point in the process this makes the most sense, but since this is still only
an RFC I've abstained from doing so yet.
Elizabeth Figura (9):
ntsync: Introduce the ntsync driver and character device.
ntsync: Reserve a minor device number and ioctl range.
ntsync: Introduce NTSYNC_IOC_CREATE_SEM and NTSYNC_IOC_DELETE.
ntsync: Introduce NTSYNC_IOC_PUT_SEM.
ntsync: Introduce NTSYNC_IOC_WAIT_ANY.
ntsync: Introduce NTSYNC_IOC_WAIT_ALL.
ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX.
ntsync: Introduce NTSYNC_IOC_PUT_MUTEX.
ntsync: Introduce NTSYNC_IOC_KILL_OWNER.
Documentation/admin-guide/devices.txt | 3 +-
.../userspace-api/ioctl/ioctl-number.rst | 2 +
drivers/misc/Kconfig | 9 +
drivers/misc/Makefile | 1 +
drivers/misc/ntsync.c | 916 ++++++++++++++++++
include/linux/miscdevice.h | 1 +
include/uapi/linux/ntsync.h | 53 +
7 files changed, 984 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/ntsync.c
create mode 100644 include/uapi/linux/ntsync.h
base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
--
2.43.0
^ permalink raw reply
* [RFC PATCH 7/9] ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX.
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
This corresponds to the NT syscall NtCreateMutant().
Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
---
drivers/misc/ntsync.c | 72 +++++++++++++++++++++++++++++++++++++
include/uapi/linux/ntsync.h | 8 +++++
2 files changed, 80 insertions(+)
diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c
index 2685363fae9e..d48f2ef41341 100644
--- a/drivers/misc/ntsync.c
+++ b/drivers/misc/ntsync.c
@@ -16,6 +16,7 @@
enum ntsync_type {
NTSYNC_TYPE_SEM,
+ NTSYNC_TYPE_MUTEX,
};
struct ntsync_obj {
@@ -60,6 +61,10 @@ struct ntsync_obj {
__u32 count;
__u32 max;
} sem;
+ struct {
+ __u32 count;
+ __u32 owner;
+ } mutex;
} u;
};
@@ -188,6 +193,10 @@ static bool is_signaled(struct ntsync_obj *obj, __u32 owner)
switch (obj->type) {
case NTSYNC_TYPE_SEM:
return !!obj->u.sem.count;
+ case NTSYNC_TYPE_MUTEX:
+ if (obj->u.mutex.owner && obj->u.mutex.owner != owner)
+ return false;
+ return obj->u.mutex.count < UINT_MAX;
}
WARN(1, "bad object type %#x\n", obj->type);
@@ -230,6 +239,10 @@ static void try_wake_all(struct ntsync_device *dev, struct ntsync_q *q,
case NTSYNC_TYPE_SEM:
obj->u.sem.count--;
break;
+ case NTSYNC_TYPE_MUTEX:
+ obj->u.mutex.count++;
+ obj->u.mutex.owner = q->owner;
+ break;
}
}
wake_up_process(q->task);
@@ -271,6 +284,28 @@ static void try_wake_any_sem(struct ntsync_obj *sem)
}
}
+static void try_wake_any_mutex(struct ntsync_obj *mutex)
+{
+ struct ntsync_q_entry *entry;
+
+ lockdep_assert_held(&mutex->lock);
+
+ list_for_each_entry(entry, &mutex->any_waiters, node) {
+ struct ntsync_q *q = entry->q;
+
+ if (mutex->u.mutex.count == UINT_MAX)
+ break;
+ if (mutex->u.mutex.owner && mutex->u.mutex.owner != q->owner)
+ continue;
+
+ if (atomic_cmpxchg(&q->signaled, -1, entry->index) == -1) {
+ mutex->u.mutex.count++;
+ mutex->u.mutex.owner = q->owner;
+ wake_up_process(q->task);
+ }
+ }
+}
+
static int ntsync_create_sem(struct ntsync_device *dev, void __user *argp)
{
struct ntsync_sem_args __user *user_args = argp;
@@ -303,6 +338,38 @@ static int ntsync_create_sem(struct ntsync_device *dev, void __user *argp)
return put_user(id, &user_args->sem);
}
+static int ntsync_create_mutex(struct ntsync_device *dev, void __user *argp)
+{
+ struct ntsync_mutex_args __user *user_args = argp;
+ struct ntsync_mutex_args args;
+ struct ntsync_obj *mutex;
+ __u32 id;
+ int ret;
+
+ if (copy_from_user(&args, argp, sizeof(args)))
+ return -EFAULT;
+
+ if (!args.owner != !args.count)
+ return -EINVAL;
+
+ mutex = kzalloc(sizeof(*mutex), GFP_KERNEL);
+ if (!mutex)
+ return -ENOMEM;
+
+ init_obj(mutex);
+ mutex->type = NTSYNC_TYPE_MUTEX;
+ mutex->u.mutex.count = args.count;
+ mutex->u.mutex.owner = args.owner;
+
+ ret = xa_alloc(&dev->objects, &id, mutex, xa_limit_32b, GFP_KERNEL);
+ if (ret < 0) {
+ kfree(mutex);
+ return ret;
+ }
+
+ return put_user(id, &user_args->mutex);
+}
+
static int ntsync_delete(struct ntsync_device *dev, void __user *argp)
{
struct ntsync_obj *obj;
@@ -497,6 +564,9 @@ static void try_wake_any_obj(struct ntsync_obj *obj)
case NTSYNC_TYPE_SEM:
try_wake_any_sem(obj);
break;
+ case NTSYNC_TYPE_MUTEX:
+ try_wake_any_mutex(obj);
+ break;
}
}
@@ -662,6 +732,8 @@ static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
void __user *argp = (void __user *)parm;
switch (cmd) {
+ case NTSYNC_IOC_CREATE_MUTEX:
+ return ntsync_create_mutex(dev, argp);
case NTSYNC_IOC_CREATE_SEM:
return ntsync_create_sem(dev, argp);
case NTSYNC_IOC_DELETE:
diff --git a/include/uapi/linux/ntsync.h b/include/uapi/linux/ntsync.h
index a5bed5a39b21..26d1b3d4847f 100644
--- a/include/uapi/linux/ntsync.h
+++ b/include/uapi/linux/ntsync.h
@@ -16,6 +16,12 @@ struct ntsync_sem_args {
__u32 max;
};
+struct ntsync_mutex_args {
+ __u32 mutex;
+ __u32 owner;
+ __u32 count;
+};
+
struct ntsync_wait_args {
__u64 timeout;
__u64 objs;
@@ -38,5 +44,7 @@ struct ntsync_wait_args {
struct ntsync_wait_args)
#define NTSYNC_IOC_WAIT_ALL _IOWR(NTSYNC_IOC_BASE, 4, \
struct ntsync_wait_args)
+#define NTSYNC_IOC_CREATE_MUTEX _IOWR(NTSYNC_IOC_BASE, 5, \
+ struct ntsync_mutex_args)
#endif
--
2.43.0
^ permalink raw reply related
* [RFC PATCH 9/9] ntsync: Introduce NTSYNC_IOC_KILL_OWNER.
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
This does not correspond to any NT syscall, but rather should be called by the
user-space NT emulator when a thread dies. It is responsible for marking any
mutexes owned by that thread as abandoned.
Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
---
drivers/misc/ntsync.c | 80 ++++++++++++++++++++++++++++++++++++-
include/uapi/linux/ntsync.h | 1 +
2 files changed, 79 insertions(+), 2 deletions(-)
diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c
index 28f43768d1c3..1173c750c106 100644
--- a/drivers/misc/ntsync.c
+++ b/drivers/misc/ntsync.c
@@ -64,6 +64,7 @@ struct ntsync_obj {
struct {
__u32 count;
__u32 owner;
+ bool ownerdead;
} mutex;
} u;
};
@@ -87,6 +88,7 @@ struct ntsync_q {
atomic_t signaled;
bool all;
+ bool ownerdead;
__u32 count;
struct ntsync_q_entry entries[];
};
@@ -240,6 +242,9 @@ static void try_wake_all(struct ntsync_device *dev, struct ntsync_q *q,
obj->u.sem.count--;
break;
case NTSYNC_TYPE_MUTEX:
+ if (obj->u.mutex.ownerdead)
+ q->ownerdead = true;
+ obj->u.mutex.ownerdead = false;
obj->u.mutex.count++;
obj->u.mutex.owner = q->owner;
break;
@@ -299,6 +304,9 @@ static void try_wake_any_mutex(struct ntsync_obj *mutex)
continue;
if (atomic_cmpxchg(&q->signaled, -1, entry->index) == -1) {
+ if (mutex->u.mutex.ownerdead)
+ q->ownerdead = true;
+ mutex->u.mutex.ownerdead = false;
mutex->u.mutex.count++;
mutex->u.mutex.owner = q->owner;
wake_up_process(q->task);
@@ -514,6 +522,71 @@ static int ntsync_put_mutex(struct ntsync_device *dev, void __user *argp)
return ret;
}
+/*
+ * Actually change the mutex state to mark its owner as dead.
+ */
+static void put_mutex_ownerdead_state(struct ntsync_obj *mutex)
+{
+ lockdep_assert_held(&mutex->lock);
+
+ mutex->u.mutex.ownerdead = true;
+ mutex->u.mutex.owner = 0;
+ mutex->u.mutex.count = 0;
+}
+
+static int ntsync_kill_owner(struct ntsync_device *dev, void __user *argp)
+{
+ struct ntsync_obj *obj;
+ unsigned long id;
+ __u32 owner;
+
+ if (get_user(owner, (__u32 __user *)argp))
+ return -EFAULT;
+ if (!owner)
+ return -EINVAL;
+
+ rcu_read_lock();
+
+ xa_for_each(&dev->objects, id, obj) {
+ if (!kref_get_unless_zero(&obj->refcount))
+ continue;
+
+ if (obj->type != NTSYNC_TYPE_MUTEX) {
+ put_obj(obj);
+ continue;
+ }
+
+ if (atomic_read(&obj->all_hint) > 0) {
+ spin_lock(&dev->wait_all_lock);
+ spin_lock_nest_lock(&obj->lock, &dev->wait_all_lock);
+
+ if (obj->u.mutex.owner == owner) {
+ put_mutex_ownerdead_state(obj);
+ try_wake_all_obj(dev, obj);
+ try_wake_any_mutex(obj);
+ }
+
+ spin_unlock(&obj->lock);
+ spin_unlock(&dev->wait_all_lock);
+ } else {
+ spin_lock(&obj->lock);
+
+ if (obj->u.mutex.owner == owner) {
+ put_mutex_ownerdead_state(obj);
+ try_wake_any_mutex(obj);
+ }
+
+ spin_unlock(&obj->lock);
+ }
+
+ put_obj(obj);
+ }
+
+ rcu_read_unlock();
+
+ return 0;
+}
+
static int ntsync_schedule(const struct ntsync_q *q, ktime_t *timeout)
{
int ret = 0;
@@ -585,6 +658,7 @@ static int setup_wait(struct ntsync_device *dev,
q->owner = args->owner;
atomic_set(&q->signaled, -1);
q->all = all;
+ q->ownerdead = false;
q->count = count;
for (i = 0; i < count; i++) {
@@ -697,7 +771,7 @@ static int ntsync_wait_any(struct ntsync_device *dev, void __user *argp)
struct ntsync_wait_args __user *user_args = argp;
/* even if we caught a signal, we need to communicate success */
- ret = 0;
+ ret = q->ownerdead ? -EOWNERDEAD : 0;
if (put_user(signaled, &user_args->index))
ret = -EFAULT;
@@ -778,7 +852,7 @@ static int ntsync_wait_all(struct ntsync_device *dev, void __user *argp)
struct ntsync_wait_args __user *user_args = argp;
/* even if we caught a signal, we need to communicate success */
- ret = 0;
+ ret = q->ownerdead ? -EOWNERDEAD : 0;
if (put_user(signaled, &user_args->index))
ret = -EFAULT;
@@ -803,6 +877,8 @@ static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
return ntsync_create_sem(dev, argp);
case NTSYNC_IOC_DELETE:
return ntsync_delete(dev, argp);
+ case NTSYNC_IOC_KILL_OWNER:
+ return ntsync_kill_owner(dev, argp);
case NTSYNC_IOC_PUT_MUTEX:
return ntsync_put_mutex(dev, argp);
case NTSYNC_IOC_PUT_SEM:
diff --git a/include/uapi/linux/ntsync.h b/include/uapi/linux/ntsync.h
index 2e44e7e77776..fec9a3993322 100644
--- a/include/uapi/linux/ntsync.h
+++ b/include/uapi/linux/ntsync.h
@@ -48,5 +48,6 @@ struct ntsync_wait_args {
struct ntsync_mutex_args)
#define NTSYNC_IOC_PUT_MUTEX _IOWR(NTSYNC_IOC_BASE, 6, \
struct ntsync_mutex_args)
+#define NTSYNC_IOC_KILL_OWNER _IOW (NTSYNC_IOC_BASE, 7, __u32)
#endif
--
2.43.0
^ permalink raw reply related
* [RFC PATCH 8/9] ntsync: Introduce NTSYNC_IOC_PUT_MUTEX.
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
This corresponds to the NT syscall NtReleaseMutant().
Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
---
drivers/misc/ntsync.c | 67 +++++++++++++++++++++++++++++++++++++
include/uapi/linux/ntsync.h | 2 ++
2 files changed, 69 insertions(+)
diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c
index d48f2ef41341..28f43768d1c3 100644
--- a/drivers/misc/ntsync.c
+++ b/drivers/misc/ntsync.c
@@ -449,6 +449,71 @@ static int ntsync_put_sem(struct ntsync_device *dev, void __user *argp)
return ret;
}
+/*
+ * Actually change the mutex state, returning -EPERM if not the owner.
+ */
+static int put_mutex_state(struct ntsync_obj *mutex,
+ const struct ntsync_mutex_args *args)
+{
+ lockdep_assert_held(&mutex->lock);
+
+ if (mutex->u.mutex.owner != args->owner)
+ return -EPERM;
+
+ if (!--mutex->u.mutex.count)
+ mutex->u.mutex.owner = 0;
+ return 0;
+}
+
+static int ntsync_put_mutex(struct ntsync_device *dev, void __user *argp)
+{
+ struct ntsync_mutex_args __user *user_args = argp;
+ struct ntsync_mutex_args args;
+ struct ntsync_obj *mutex;
+ __u32 prev_count;
+ int ret;
+
+ if (copy_from_user(&args, argp, sizeof(args)))
+ return -EFAULT;
+ if (!args.owner)
+ return -EINVAL;
+
+ mutex = get_obj_typed(dev, args.mutex, NTSYNC_TYPE_MUTEX);
+ if (!mutex)
+ return -EINVAL;
+
+ if (atomic_read(&mutex->all_hint) > 0) {
+ spin_lock(&dev->wait_all_lock);
+ spin_lock_nest_lock(&mutex->lock, &dev->wait_all_lock);
+
+ prev_count = mutex->u.mutex.count;
+ ret = put_mutex_state(mutex, &args);
+ if (!ret) {
+ try_wake_all_obj(dev, mutex);
+ try_wake_any_mutex(mutex);
+ }
+
+ spin_unlock(&mutex->lock);
+ spin_unlock(&dev->wait_all_lock);
+ } else {
+ spin_lock(&mutex->lock);
+
+ prev_count = mutex->u.mutex.count;
+ ret = put_mutex_state(mutex, &args);
+ if (!ret)
+ try_wake_any_mutex(mutex);
+
+ spin_unlock(&mutex->lock);
+ }
+
+ put_obj(mutex);
+
+ if (!ret && put_user(prev_count, &user_args->count))
+ ret = -EFAULT;
+
+ return ret;
+}
+
static int ntsync_schedule(const struct ntsync_q *q, ktime_t *timeout)
{
int ret = 0;
@@ -738,6 +803,8 @@ static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
return ntsync_create_sem(dev, argp);
case NTSYNC_IOC_DELETE:
return ntsync_delete(dev, argp);
+ case NTSYNC_IOC_PUT_MUTEX:
+ return ntsync_put_mutex(dev, argp);
case NTSYNC_IOC_PUT_SEM:
return ntsync_put_sem(dev, argp);
case NTSYNC_IOC_WAIT_ALL:
diff --git a/include/uapi/linux/ntsync.h b/include/uapi/linux/ntsync.h
index 26d1b3d4847f..2e44e7e77776 100644
--- a/include/uapi/linux/ntsync.h
+++ b/include/uapi/linux/ntsync.h
@@ -46,5 +46,7 @@ struct ntsync_wait_args {
struct ntsync_wait_args)
#define NTSYNC_IOC_CREATE_MUTEX _IOWR(NTSYNC_IOC_BASE, 5, \
struct ntsync_mutex_args)
+#define NTSYNC_IOC_PUT_MUTEX _IOWR(NTSYNC_IOC_BASE, 6, \
+ struct ntsync_mutex_args)
#endif
--
2.43.0
^ permalink raw reply related
* [RFC PATCH 2/9] ntsync: Reserve a minor device number and ioctl range.
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
---
Documentation/admin-guide/devices.txt | 3 ++-
Documentation/userspace-api/ioctl/ioctl-number.rst | 2 ++
drivers/misc/ntsync.c | 3 ++-
include/linux/miscdevice.h | 1 +
4 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/devices.txt b/Documentation/admin-guide/devices.txt
index 94c98be1329a..041404397ee5 100644
--- a/Documentation/admin-guide/devices.txt
+++ b/Documentation/admin-guide/devices.txt
@@ -376,8 +376,9 @@
240 = /dev/userio Serio driver testing device
241 = /dev/vhost-vsock Host kernel driver for virtio vsock
242 = /dev/rfkill Turning off radio transmissions (rfkill)
+ 243 = /dev/ntsync NT synchronization primitive device
- 243-254 Reserved for local use
+ 244-254 Reserved for local use
255 Reserved for MISC_DYNAMIC_MINOR
11 char Raw keyboard device (Linux/SPARC only)
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 457e16f06e04..a1326a5bc2e0 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -378,6 +378,8 @@ Code Seq# Include File Comments
<mailto:thomas@winischhofer.net>
0xF6 all LTTng Linux Trace Toolkit Next Generation
<mailto:mathieu.desnoyers@efficios.com>
+0xF7 00-1F uapi/linux/ntsync.h NT synchronization primitives
+ <mailto:wine-devel@winehq.org>
0xF8 all arch/x86/include/uapi/asm/amd_hsmp.h AMD HSMP EPYC system management interface driver
<mailto:nchatrad@amd.com>
0xFD all linux/dm-ioctl.h
diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c
index 9424c6210e51..84b498e2b2d5 100644
--- a/drivers/misc/ntsync.c
+++ b/drivers/misc/ntsync.c
@@ -40,7 +40,7 @@ static const struct file_operations ntsync_fops = {
};
static struct miscdevice ntsync_misc = {
- .minor = MISC_DYNAMIC_MINOR,
+ .minor = NTSYNC_MINOR,
.name = NTSYNC_NAME,
.fops = &ntsync_fops,
};
@@ -51,3 +51,4 @@ MODULE_AUTHOR("Elizabeth Figura");
MODULE_DESCRIPTION("Kernel driver for NT synchronization primitives");
MODULE_LICENSE("GPL");
MODULE_ALIAS("devname:" NTSYNC_NAME);
+MODULE_ALIAS_MISCDEV(NTSYNC_MINOR);
diff --git a/include/linux/miscdevice.h b/include/linux/miscdevice.h
index c0fea6ca5076..fe5d9366fdf7 100644
--- a/include/linux/miscdevice.h
+++ b/include/linux/miscdevice.h
@@ -71,6 +71,7 @@
#define USERIO_MINOR 240
#define VHOST_VSOCK_MINOR 241
#define RFKILL_MINOR 242
+#define NTSYNC_MINOR 243
#define MISC_DYNAMIC_MINOR 255
struct device;
--
2.43.0
^ permalink raw reply related
* [RFC PATCH 5/9] ntsync: Introduce NTSYNC_IOC_WAIT_ANY.
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
This corresponds to part of the functionality of the NT syscall
NtWaitForMultipleObjects(). Specifically, it implements the behaviour where
the third argument (wait_any) is TRUE, and it does not handle alertable waits.
Those features have been split out into separate patches to ease review.
Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
---
drivers/misc/ntsync.c | 229 ++++++++++++++++++++++++++++++++++++
include/uapi/linux/ntsync.h | 13 ++
2 files changed, 242 insertions(+)
diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c
index d1c91c2a4f1a..2e8d3c2d51a4 100644
--- a/drivers/misc/ntsync.c
+++ b/drivers/misc/ntsync.c
@@ -23,6 +23,8 @@ struct ntsync_obj {
struct kref refcount;
spinlock_t lock;
+ struct list_head any_waiters;
+
enum ntsync_type type;
/* The following fields are protected by the object lock. */
@@ -34,6 +36,28 @@ struct ntsync_obj {
} u;
};
+struct ntsync_q_entry {
+ struct list_head node;
+ struct ntsync_q *q;
+ struct ntsync_obj *obj;
+ __u32 index;
+};
+
+struct ntsync_q {
+ struct task_struct *task;
+ __u32 owner;
+
+ /*
+ * Protected via atomic_cmpxchg(). Only the thread that wins the
+ * compare-and-swap may actually change object states and wake this
+ * task.
+ */
+ atomic_t signaled;
+
+ __u32 count;
+ struct ntsync_q_entry entries[];
+};
+
struct ntsync_device {
struct xarray objects;
};
@@ -109,6 +133,26 @@ static void init_obj(struct ntsync_obj *obj)
{
kref_init(&obj->refcount);
spin_lock_init(&obj->lock);
+ INIT_LIST_HEAD(&obj->any_waiters);
+}
+
+static void try_wake_any_sem(struct ntsync_obj *sem)
+{
+ struct ntsync_q_entry *entry;
+
+ lockdep_assert_held(&sem->lock);
+
+ list_for_each_entry(entry, &sem->any_waiters, node) {
+ struct ntsync_q *q = entry->q;
+
+ if (!sem->u.sem.count)
+ break;
+
+ if (atomic_cmpxchg(&q->signaled, -1, entry->index) == -1) {
+ sem->u.sem.count--;
+ wake_up_process(q->task);
+ }
+ }
}
static int ntsync_create_sem(struct ntsync_device *dev, void __user *argp)
@@ -194,6 +238,8 @@ static int ntsync_put_sem(struct ntsync_device *dev, void __user *argp)
prev_count = sem->u.sem.count;
ret = put_sem_state(sem, args.count);
+ if (!ret)
+ try_wake_any_sem(sem);
spin_unlock(&sem->lock);
@@ -205,6 +251,187 @@ static int ntsync_put_sem(struct ntsync_device *dev, void __user *argp)
return ret;
}
+static int ntsync_schedule(const struct ntsync_q *q, ktime_t *timeout)
+{
+ int ret = 0;
+
+ do {
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
+
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (atomic_read(&q->signaled) != -1) {
+ ret = 0;
+ break;
+ }
+ ret = schedule_hrtimeout(timeout, HRTIMER_MODE_ABS);
+ } while (ret < 0);
+ __set_current_state(TASK_RUNNING);
+
+ return ret;
+}
+
+/*
+ * Allocate and initialize the ntsync_q structure, but do not queue us yet.
+ * Also, calculate the relative timeout.
+ */
+static int setup_wait(struct ntsync_device *dev,
+ const struct ntsync_wait_args *args,
+ ktime_t *ret_timeout, struct ntsync_q **ret_q)
+{
+ const __u32 count = args->count;
+ struct ntsync_q *q;
+ ktime_t timeout = 0;
+ __u32 *ids;
+ __u32 i, j;
+
+ if (!args->owner || args->pad)
+ return -EINVAL;
+
+ if (args->count > NTSYNC_MAX_WAIT_COUNT)
+ return -EINVAL;
+
+ if (args->timeout) {
+ struct timespec64 to;
+
+ if (get_timespec64(&to, u64_to_user_ptr(args->timeout)))
+ return -EFAULT;
+ if (!timespec64_valid(&to))
+ return -EINVAL;
+
+ timeout = timespec64_to_ns(&to);
+ }
+
+ ids = kmalloc_array(count, sizeof(*ids), GFP_KERNEL);
+ if (!ids)
+ return -ENOMEM;
+ if (copy_from_user(ids, u64_to_user_ptr(args->objs),
+ array_size(count, sizeof(*ids)))) {
+ kfree(ids);
+ return -EFAULT;
+ }
+
+ q = kmalloc(struct_size(q, entries, count), GFP_KERNEL);
+ if (!q) {
+ kfree(ids);
+ return -ENOMEM;
+ }
+ q->task = current;
+ q->owner = args->owner;
+ atomic_set(&q->signaled, -1);
+ q->count = count;
+
+ for (i = 0; i < count; i++) {
+ struct ntsync_q_entry *entry = &q->entries[i];
+ struct ntsync_obj *obj = get_obj(dev, ids[i]);
+
+ if (!obj)
+ goto err;
+
+ entry->obj = obj;
+ entry->q = q;
+ entry->index = i;
+ }
+
+ kfree(ids);
+
+ *ret_q = q;
+ *ret_timeout = timeout;
+ return 0;
+
+err:
+ for (j = 0; j < i; j++)
+ put_obj(q->entries[j].obj);
+ kfree(ids);
+ kfree(q);
+ return -EINVAL;
+}
+
+static void try_wake_any_obj(struct ntsync_obj *obj)
+{
+ switch (obj->type) {
+ case NTSYNC_TYPE_SEM:
+ try_wake_any_sem(obj);
+ break;
+ }
+}
+
+static int ntsync_wait_any(struct ntsync_device *dev, void __user *argp)
+{
+ struct ntsync_wait_args args;
+ struct ntsync_q *q;
+ ktime_t timeout;
+ int signaled;
+ __u32 i;
+ int ret;
+
+ if (copy_from_user(&args, argp, sizeof(args)))
+ return -EFAULT;
+
+ ret = setup_wait(dev, &args, &timeout, &q);
+ if (ret < 0)
+ return ret;
+
+ /* queue ourselves */
+
+ for (i = 0; i < args.count; i++) {
+ struct ntsync_q_entry *entry = &q->entries[i];
+ struct ntsync_obj *obj = entry->obj;
+
+ spin_lock(&obj->lock);
+ list_add_tail(&entry->node, &obj->any_waiters);
+ spin_unlock(&obj->lock);
+ }
+
+ /* check if we are already signaled */
+
+ for (i = 0; i < args.count; i++) {
+ struct ntsync_obj *obj = q->entries[i].obj;
+
+ if (atomic_read(&q->signaled) != -1)
+ break;
+
+ spin_lock(&obj->lock);
+ try_wake_any_obj(obj);
+ spin_unlock(&obj->lock);
+ }
+
+ /* sleep */
+
+ ret = ntsync_schedule(q, args.timeout ? &timeout : NULL);
+
+ /* and finally, unqueue */
+
+ for (i = 0; i < args.count; i++) {
+ struct ntsync_q_entry *entry = &q->entries[i];
+ struct ntsync_obj *obj = entry->obj;
+
+ spin_lock(&obj->lock);
+ list_del(&entry->node);
+ spin_unlock(&obj->lock);
+
+ put_obj(obj);
+ }
+
+ signaled = atomic_read(&q->signaled);
+ if (signaled != -1) {
+ struct ntsync_wait_args __user *user_args = argp;
+
+ /* even if we caught a signal, we need to communicate success */
+ ret = 0;
+
+ if (put_user(signaled, &user_args->index))
+ ret = -EFAULT;
+ } else if (!ret) {
+ ret = -ETIMEDOUT;
+ }
+
+ kfree(q);
+ return ret;
+}
+
static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
unsigned long parm)
{
@@ -218,6 +445,8 @@ static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
return ntsync_delete(dev, argp);
case NTSYNC_IOC_PUT_SEM:
return ntsync_put_sem(dev, argp);
+ case NTSYNC_IOC_WAIT_ANY:
+ return ntsync_wait_any(dev, argp);
default:
return -ENOIOCTLCMD;
}
diff --git a/include/uapi/linux/ntsync.h b/include/uapi/linux/ntsync.h
index 8c610d65f8ef..10f07da7864e 100644
--- a/include/uapi/linux/ntsync.h
+++ b/include/uapi/linux/ntsync.h
@@ -16,6 +16,17 @@ struct ntsync_sem_args {
__u32 max;
};
+struct ntsync_wait_args {
+ __u64 timeout;
+ __u64 objs;
+ __u32 count;
+ __u32 owner;
+ __u32 index;
+ __u32 pad;
+};
+
+#define NTSYNC_MAX_WAIT_COUNT 64
+
#define NTSYNC_IOC_BASE 0xf7
#define NTSYNC_IOC_CREATE_SEM _IOWR(NTSYNC_IOC_BASE, 0, \
@@ -23,5 +34,7 @@ struct ntsync_sem_args {
#define NTSYNC_IOC_DELETE _IOW (NTSYNC_IOC_BASE, 1, __u32)
#define NTSYNC_IOC_PUT_SEM _IOWR(NTSYNC_IOC_BASE, 2, \
struct ntsync_sem_args)
+#define NTSYNC_IOC_WAIT_ANY _IOWR(NTSYNC_IOC_BASE, 3, \
+ struct ntsync_wait_args)
#endif
--
2.43.0
^ permalink raw reply related
* [RFC PATCH 1/9] ntsync: Introduce the ntsync driver and character device.
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
ntsync uses a misc device as the simplest and least intrusive uAPI interface.
Each file description on the device represents an isolated NT instance, intended
to correspond to a single NT virtual machine.
Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
---
drivers/misc/Kconfig | 9 ++++++++
drivers/misc/Makefile | 1 +
drivers/misc/ntsync.c | 53 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 63 insertions(+)
create mode 100644 drivers/misc/ntsync.c
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index 4fb291f0bf7c..bdd8a71bd853 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -504,6 +504,15 @@ config OPEN_DICE
measured boot flow. Userspace can use CDIs for remote attestation
and sealing.
+config NTSYNC
+ tristate "NT synchronization primitive emulation"
+ help
+ This module provides kernel support for emulation of Windows NT
+ synchronization primitives. It is not a hardware driver.
+
+ To compile this driver as a module, choose M here: the
+ module will be called ntsync.
+
If unsure, say N.
config VCPU_STALL_DETECTOR
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index ea6ea5bbbc9c..153a3f4837e8 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -59,6 +59,7 @@ obj-$(CONFIG_PVPANIC) += pvpanic/
obj-$(CONFIG_UACCE) += uacce/
obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o
obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o
+obj-$(CONFIG_NTSYNC) += ntsync.o
obj-$(CONFIG_HI6421V600_IRQ) += hi6421v600-irq.o
obj-$(CONFIG_OPEN_DICE) += open-dice.o
obj-$(CONFIG_GP_PCI1XXXX) += mchp_pci1xxxx/
diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c
new file mode 100644
index 000000000000..9424c6210e51
--- /dev/null
+++ b/drivers/misc/ntsync.c
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * ntsync.c - Kernel driver for NT synchronization primitives
+ *
+ * Copyright (C) 2021-2022 Elizabeth Figura
+ */
+
+#include <linux/fs.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+
+#define NTSYNC_NAME "ntsync"
+
+static int ntsync_char_open(struct inode *inode, struct file *file)
+{
+ return nonseekable_open(inode, file);
+}
+
+static int ntsync_char_release(struct inode *inode, struct file *file)
+{
+ return 0;
+}
+
+static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
+ unsigned long parm)
+{
+ switch (cmd) {
+ default:
+ return -ENOIOCTLCMD;
+ }
+}
+
+static const struct file_operations ntsync_fops = {
+ .owner = THIS_MODULE,
+ .open = ntsync_char_open,
+ .release = ntsync_char_release,
+ .unlocked_ioctl = ntsync_char_ioctl,
+ .compat_ioctl = ntsync_char_ioctl,
+ .llseek = no_llseek,
+};
+
+static struct miscdevice ntsync_misc = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = NTSYNC_NAME,
+ .fops = &ntsync_fops,
+};
+
+module_misc_device(ntsync_misc);
+
+MODULE_AUTHOR("Elizabeth Figura");
+MODULE_DESCRIPTION("Kernel driver for NT synchronization primitives");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("devname:" NTSYNC_NAME);
--
2.43.0
^ permalink raw reply related
* [RFC PATCH 4/9] ntsync: Introduce NTSYNC_IOC_PUT_SEM.
From: Elizabeth Figura @ 2024-01-24 0:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, linux-kernel, linux-api
Cc: wine-devel, André Almeida, Wolfram Sang, Arkadiusz Hiler,
Peter Zijlstra, Elizabeth Figura
In-Reply-To: <20240124004028.16826-1-zfigura@codeweavers.com>
This corresponds to the NT syscall NtReleaseSemaphore().
Signed-off-by: Elizabeth Figura <zfigura@codeweavers.com>
---
drivers/misc/ntsync.c | 76 +++++++++++++++++++++++++++++++++++++
include/uapi/linux/ntsync.h | 2 +
2 files changed, 78 insertions(+)
diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c
index 3287b94be351..d1c91c2a4f1a 100644
--- a/drivers/misc/ntsync.c
+++ b/drivers/misc/ntsync.c
@@ -21,9 +21,11 @@ enum ntsync_type {
struct ntsync_obj {
struct rcu_head rhead;
struct kref refcount;
+ spinlock_t lock;
enum ntsync_type type;
+ /* The following fields are protected by the object lock. */
union {
struct {
__u32 count;
@@ -36,6 +38,19 @@ struct ntsync_device {
struct xarray objects;
};
+static struct ntsync_obj *get_obj(struct ntsync_device *dev, __u32 id)
+{
+ struct ntsync_obj *obj;
+
+ rcu_read_lock();
+ obj = xa_load(&dev->objects, id);
+ if (obj && !kref_get_unless_zero(&obj->refcount))
+ obj = NULL;
+ rcu_read_unlock();
+
+ return obj;
+}
+
static void destroy_obj(struct kref *ref)
{
struct ntsync_obj *obj = container_of(ref, struct ntsync_obj, refcount);
@@ -48,6 +63,18 @@ static void put_obj(struct ntsync_obj *obj)
kref_put(&obj->refcount, destroy_obj);
}
+static struct ntsync_obj *get_obj_typed(struct ntsync_device *dev, __u32 id,
+ enum ntsync_type type)
+{
+ struct ntsync_obj *obj = get_obj(dev, id);
+
+ if (obj && obj->type != type) {
+ put_obj(obj);
+ return NULL;
+ }
+ return obj;
+}
+
static int ntsync_char_open(struct inode *inode, struct file *file)
{
struct ntsync_device *dev;
@@ -81,6 +108,7 @@ static int ntsync_char_release(struct inode *inode, struct file *file)
static void init_obj(struct ntsync_obj *obj)
{
kref_init(&obj->refcount);
+ spin_lock_init(&obj->lock);
}
static int ntsync_create_sem(struct ntsync_device *dev, void __user *argp)
@@ -131,6 +159,52 @@ static int ntsync_delete(struct ntsync_device *dev, void __user *argp)
return 0;
}
+/*
+ * Actually change the semaphore state, returning -EOVERFLOW if it is made
+ * invalid.
+ */
+static int put_sem_state(struct ntsync_obj *sem, __u32 count)
+{
+ lockdep_assert_held(&sem->lock);
+
+ if (sem->u.sem.count + count < sem->u.sem.count ||
+ sem->u.sem.count + count > sem->u.sem.max)
+ return -EOVERFLOW;
+
+ sem->u.sem.count += count;
+ return 0;
+}
+
+static int ntsync_put_sem(struct ntsync_device *dev, void __user *argp)
+{
+ struct ntsync_sem_args __user *user_args = argp;
+ struct ntsync_sem_args args;
+ struct ntsync_obj *sem;
+ __u32 prev_count;
+ int ret;
+
+ if (copy_from_user(&args, argp, sizeof(args)))
+ return -EFAULT;
+
+ sem = get_obj_typed(dev, args.sem, NTSYNC_TYPE_SEM);
+ if (!sem)
+ return -EINVAL;
+
+ spin_lock(&sem->lock);
+
+ prev_count = sem->u.sem.count;
+ ret = put_sem_state(sem, args.count);
+
+ spin_unlock(&sem->lock);
+
+ put_obj(sem);
+
+ if (!ret && put_user(prev_count, &user_args->count))
+ ret = -EFAULT;
+
+ return ret;
+}
+
static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
unsigned long parm)
{
@@ -142,6 +216,8 @@ static long ntsync_char_ioctl(struct file *file, unsigned int cmd,
return ntsync_create_sem(dev, argp);
case NTSYNC_IOC_DELETE:
return ntsync_delete(dev, argp);
+ case NTSYNC_IOC_PUT_SEM:
+ return ntsync_put_sem(dev, argp);
default:
return -ENOIOCTLCMD;
}
diff --git a/include/uapi/linux/ntsync.h b/include/uapi/linux/ntsync.h
index d97afc138dcc..8c610d65f8ef 100644
--- a/include/uapi/linux/ntsync.h
+++ b/include/uapi/linux/ntsync.h
@@ -21,5 +21,7 @@ struct ntsync_sem_args {
#define NTSYNC_IOC_CREATE_SEM _IOWR(NTSYNC_IOC_BASE, 0, \
struct ntsync_sem_args)
#define NTSYNC_IOC_DELETE _IOW (NTSYNC_IOC_BASE, 1, __u32)
+#define NTSYNC_IOC_PUT_SEM _IOWR(NTSYNC_IOC_BASE, 2, \
+ struct ntsync_sem_args)
#endif
--
2.43.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox