Re: [PATCH RFC 00/25] Replacing net_mutex with rw_semaphore

[ebiederm@xmission.com (Eric W. Biederman)]

Kirill Tkhai <ktkhai@virtuozzo.com> writes:

> Hi,
>
> this is continuation of discussion from here:
>
> https://lkml.org/lkml/2017/11/14/298
>
> The plan has changed a little bit, so I'd be happy to hear
> people's comments, before I dived into all 400+ pernet subsys
> and devices.
>
> The patch set adds pernet sys list ahead of subsys and device,
> and it's used for pernet_operations, which may be executed
> in parallel with any other pernet_operations methods. Also,
> some high-priority ops converted (up to registered using
> postcore_initcall(), and some subsys_initcall()) in order
> of appearance. The sequence in setup_net() is following:
>
> 1)execute all the callbacks from pernet_sys list
> 2)lock net_mutex
> 3)execute all the callbacks from pernet_subsys list
> 4)execute all the callbacks from pernet_device list
> 5)unlock net_mutex
>
> There was not pernet_operations, requiring additional
> synchronization, yet, but I've bumped in another problem.
> The problem is that some drivers may be compiled as modules
> and as kernel-image part. They register pernet_operations
> from device_initcall() for example. This initcall executes
> in different time comparing to in-kernel built-in only
> drivers.
>
> Imagine, we have three state driverA, and boolean driverB.
> driverA registers pernet_subsys from subsys_initcall().
> driverB registers pernet_subsys from fs_initcall().
> So, here we have two cases:
>
> driverA is module              driverA is built-in
> --------------------           -------------------
> register driverB ops           register driverA ops
> register driverA ops           register driverB ops
>
> So, the order is different. When converting driver one-by-one,
> it's impossible to make the order true for all .config
> states, because of the above. So, the bisect won't work.
>
> And it seems, it's just the same as to convert pernet_operations
> from all the files in file alphabetical order. What do you
> think about this? (Note, the patches has no such a problem
> at the moment, as there are all in-kernel early core drivers).
>
> Maybe there are another comments on the code.

I think there is an solution in the center.  Just have a count
(protected by down_write net_sem) of the number of modules that still
need net_mutex.

If the count is non-zero we take net_mutex in setup_net and cleanup_net.

That way limited network stacks can see the benefit and actively test
the parallism, while other configurations can be safe in still having
the same behavior.

Eric


> ---
>
> Kirill Tkhai (25):
>       net: Assign net to net_namespace_list in setup_net()
>       net: Cleanup copy_net_ns()
>       net: Introduce net_sem for protection of pernet_list
>       net: Move mutex_unlock() in cleanup_net() up
>       net: Add primitives to update heads of pernet_list sublists
>       net: Add pernet sys and registration functions
>       net: Make sys sublist pernet_operations executed out of net_mutex
>       net: Move proc_net_ns_ops to pernet_sys list
>       net: Move net_ns_ops to pernet_sys list
>       net: Move sysctl_pernet_ops to pernet_sys list
>       net: Move netfilter_net_ops to pernet_sys list
>       net: Move nf_log_net_ops to pernet_sys list
>       net: Move net_inuse_ops to pernet_sys list
>       net: Move net_defaults_ops to pernet_sys list
>       net: Move netlink_net_ops to pernet_sys list
>       net: Move rtnetlink_net_ops to pernet_sys list
>       net: Move audit_net_ops to pernet_sys list
>       net: Move uevent_net_ops to pernet_sys list
>       net: Move proto_net_ops to pernet_sys list
>       net: Move pernet_subsys, registered via net_dev_init(), to pernet_sys list
>       net: Move fib_* pernet_operations, registered via subsys_initcall(), to pernet_sys list
>       net: Move subsys_initcall() registered pernet_operations from net/sched to pernet_sys list
>       net: Move genl_pernet_ops to pernet_sys list
>       net: Move wext_pernet_ops to pernet_sys list
>       net: Move sysctl_core_ops to pernet_sys list
>
>
>  fs/proc/proc_net.c          |    2 
>  include/linux/rtnetlink.h   |    1 
>  include/net/net_namespace.h |    2 
>  kernel/audit.c              |    2 
>  lib/kobject_uevent.c        |    2 
>  net/core/dev.c              |    2 
>  net/core/fib_notifier.c     |    2 
>  net/core/fib_rules.c        |    2 
>  net/core/net-procfs.c       |    4 -
>  net/core/net_namespace.c    |  203 +++++++++++++++++++++++++++++++++----------
>  net/core/rtnetlink.c        |    6 +
>  net/core/sock.c             |    4 -
>  net/core/sysctl_net_core.c  |    2 
>  net/netfilter/core.c        |    2 
>  net/netfilter/nf_log.c      |    2 
>  net/netlink/af_netlink.c    |    2 
>  net/netlink/genetlink.c     |    2 
>  net/sched/act_api.c         |    2 
>  net/sched/sch_api.c         |    2 
>  net/sysctl_net.c            |    2 
>  net/wireless/wext-core.c    |    2 
>  21 files changed, 183 insertions(+), 67 deletions(-)
>
> --
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>