Re: [PATCH RFC 00/25] Replacing net_mutex with rw_semaphore

Eric W. Biederman
2017-11-19T01:52:09Z

Kirill Tkhai <ktkhai@virtuozzo.com> writes:

> Hi,
>
> this is continuation of discussion from here:
>
> https://lkml.org/lkml/2017/11/14/298
>
> The plan has changed a little bit, so I'd be happy to hear
> people's comments, before I dived into all 400+ pernet subsys
> and devices.
>
> The patch set adds pernet sys list ahead of subsys and device,
> and it's used for pernet_operations, which may be executed
> in parallel with any other pernet_operations methods. Also,
> some high-priority ops converted (up to registered using
> postcore_initcall(), and some subsys_initcall()) in order
> of appearance. The sequence in setup_net() is following:
>
> 1)execute all the callbacks from pernet_sys list
> 2)lock net_mutex
> 3)execute all the callbacks from pernet_subsys list
> 4)execute all the callbacks from pernet_device list
> 5)unlock net_mutex
>
> There was not pernet_operations, requiring additional
> synchronization, yet, but I've bumped in another problem.
> The problem is that some drivers may be compiled as modules
> and as kernel-image part. They register pernet_operations
> from device_initcall() for example. This initcall executes
> in different time comparing to in-kernel built-in only
> drivers.
>
> Imagine, we have three state driverA, and boolean driverB.
> driverA registers pernet_subsys from subsys_initcall().
> driverB registers pernet_subsys from fs_initcall().
> So, here we have two cases:
>
> driverA is module driverA is built-in
> -------------------- -------------------
> register driverB ops register driverA ops
> register driverA ops register driverB ops
>
> So, the order is different. When converting driver one-by-one,
> it's impossible to make the order true for all .config
> states, because of the above. So, the bisect won't work.
>
> And it seems, it's just the same as to convert pernet_operations
> from all the files in file alphabetical order. What do you
> think about this? (Note, the patches has no such a problem
> at the moment, as there are all in-kernel early core drivers).
>
> Maybe there are another comments on the code.

I think there is an solution in the center. Just have a count
(protected by down_write net_sem) of the number of modules that still
need net_mutex.

If the count is non-zero we take net_mutex in setup_net and cleanup_net.

That way limited network stacks can see the benefit and actively test
the parallism, while other configurations can be safe in still having
the same behavior.

Eric

> ---
>
> Kirill Tkhai (25):
> net: Assign net to net_namespace_list in setup_net()
> net: Cleanup copy_net_ns()
> net: Introduce net_sem for protection of pernet_list
> net: Move mutex_unlock() in cleanup_net() up
> net: Add primitives to update heads of pernet_list sublists
> net: Add pernet sys and registration functions
> net: Make sys sublist pernet_operations executed out of net_mutex
> net: Move proc_net_ns_ops to pernet_sys list
> net: Move net_ns_ops to pernet_sys list
> net: Move sysctl_pernet_ops to pernet_sys list
> net: Move netfilter_net_ops to pernet_sys list
> net: Move nf_log_net_ops to pernet_sys list
> net: Move net_inuse_ops to pernet_sys list
> net: Move net_defaults_ops to pernet_sys list
> net: Move netlink_net_ops to pernet_sys list
> net: Move rtnetlink_net_ops to pernet_sys list
> net: Move audit_net_ops to pernet_sys list
> net: Move uevent_net_ops to pernet_sys list
> net: Move proto_net_ops to pernet_sys list
> net: Move pernet_subsys, registered via net_dev_init(), to pernet_sys list
> net: Move fib_* pernet_operations, registered via subsys_initcall(), to pernet_sys list
> net: Move subsys_initcall() registered pernet_operations from net/sched to pernet_sys list
> net: Move genl_pernet_ops to pernet_sys list
> net: Move wext_pernet_ops to pernet_sys list
> net: Move sysctl_core_ops to pernet_sys list
>
>
> fs/proc/proc_net.c | 2
> include/linux/rtnetlink.h | 1
> include/net/net_namespace.h | 2
> kernel/audit.c | 2
> lib/kobject_uevent.c | 2
> net/core/dev.c | 2
> net/core/fib_notifier.c | 2
> net/core/fib_rules.c | 2
> net/core/net-procfs.c | 4 -
> net/core/net_namespace.c | 203 +++++++++++++++++++++++++++++++++----------
> net/core/rtnetlink.c | 6 +
> net/core/sock.c | 4 -
> net/core/sysctl_net_core.c | 2
> net/netfilter/core.c | 2
> net/netfilter/nf_log.c | 2
> net/netlink/af_netlink.c | 2
> net/netlink/genetlink.c | 2
> net/sched/act_api.c | 2
> net/sched/sch_api.c | 2
> net/sysctl_net.c | 2
> net/wireless/wext-core.c | 2
> 21 files changed, 183 insertions(+), 67 deletions(-)
>
> --
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>