udev kset_create_and_add() with own struct kobj_type?

udev kset_create_and_add() with own struct kobj_type?

[Alexander Aring]

Hi,

I am currently implementing namespace support into the fs/dlm
subsystem. The DLM subsystem uses kset_create_and_add("dlm",
&dlm_uevent_ops, kernel_kobj); to create the "dlm" sysfs directory.
Underneath it creates for each lockspace a subdirectory using
kobject_init_and_add() with a non-static name and being known during
runtime.

Now I want to add namespace support and need to change the "default"
kset->kobj.ktype = &kset_ktype; that is set by
kset_create_and_add()->kset_create() to my own kobj_type because I
need to implement a different ".child_ns_type" callback before
kset_register() is called.

So kset_create_and_add() does not allow me to add my own
".child_ns_type" callback for the kset that is required for me to have
my sysfs "kernel/dlm" directory separated by each net-namespace
without breaking any backwards compatibility.

My current solution is that I mostly copy&pasted the code of
kset_create_and_add()/kset_create() to have a way to tell my own
struct kobj_type that contains the implemented my own ".child_ns_type"
callback.

I am writing this mail to get some advice on what I can do here
without doing the copy&pasted code?
Add a parameter to kset_create_and_add() that allows me to set an
"struct kobj_type"?
Introducing a new function that does allow me to set the new parameter
(I probably like that because then I don't need to update all other
users)?

Thanks in advance for any advice.

- Alex

Re: udev kset_create_and_add() with own struct kobj_type?

[Greg KH]

On Mon, Jul 15, 2024 at 11:27:00AM -0400, Alexander Aring wrote:
> Hi,
> 
> I am currently implementing namespace support into the fs/dlm
> subsystem. The DLM subsystem uses kset_create_and_add("dlm",
> &dlm_uevent_ops, kernel_kobj); to create the "dlm" sysfs directory.
> Underneath it creates for each lockspace a subdirectory using
> kobject_init_and_add() with a non-static name and being known during
> runtime.
> 
> Now I want to add namespace support and need to change the "default"
> kset->kobj.ktype = &kset_ktype; that is set by
> kset_create_and_add()->kset_create() to my own kobj_type because I
> need to implement a different ".child_ns_type" callback before
> kset_register() is called.
> 
> So kset_create_and_add() does not allow me to add my own
> ".child_ns_type" callback for the kset that is required for me to have
> my sysfs "kernel/dlm" directory separated by each net-namespace
> without breaking any backwards compatibility.

I don't understand, what "backwards compatibility" is happening here?

> My current solution is that I mostly copy&pasted the code of
> kset_create_and_add()/kset_create() to have a way to tell my own
> struct kobj_type that contains the implemented my own ".child_ns_type"
> callback.

Ick.

> I am writing this mail to get some advice on what I can do here
> without doing the copy&pasted code?
> Add a parameter to kset_create_and_add() that allows me to set an
> "struct kobj_type"?
> Introducing a new function that does allow me to set the new parameter
> (I probably like that because then I don't need to update all other
> users)?

A new function would be ok, but I hate it how filesystems have to use
"raw" kobjects and the like all the time.  It's rough, sorry.

Also, I didn't think sysfs namespaces worked for anything except
networking stuff.  Are you sure you really need this?  Where is your
"namespace" defined at?

thanks,

greg k-h

Re: udev kset_create_and_add() with own struct kobj_type?

[Alexander Aring]

Hi,

On Mon, Jul 15, 2024 at 11:33 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Mon, Jul 15, 2024 at 11:27:00AM -0400, Alexander Aring wrote:
> > Hi,
> >
> > I am currently implementing namespace support into the fs/dlm
> > subsystem. The DLM subsystem uses kset_create_and_add("dlm",
> > &dlm_uevent_ops, kernel_kobj); to create the "dlm" sysfs directory.
> > Underneath it creates for each lockspace a subdirectory using
> > kobject_init_and_add() with a non-static name and being known during
> > runtime.
> >
> > Now I want to add namespace support and need to change the "default"
> > kset->kobj.ktype = &kset_ktype; that is set by
> > kset_create_and_add()->kset_create() to my own kobj_type because I
> > need to implement a different ".child_ns_type" callback before
> > kset_register() is called.
> >
> > So kset_create_and_add() does not allow me to add my own
> > ".child_ns_type" callback for the kset that is required for me to have
> > my sysfs "kernel/dlm" directory separated by each net-namespace
> > without breaking any backwards compatibility.
>
> I don't understand, what "backwards compatibility" is happening here?
>

The DLM sysfs udev directory for kobjects begins in
"/sys/kernel/dlm/", that is the created kset by kset_create_and_add().
Then a lot of per lockspace kobject directories are created by
kobject_init_and_add() with a name known during runtime only.
I cannot simply do a "/sys/kernel/dlm/lockspaces/" and have
"lockspaces" separated by namespaces, it needs to be "dlm" as user
space assume a specific directory structure in "/sys/kernel/dlm" and
interprets it and we want to have lockspaces separated per namespace.

> > My current solution is that I mostly copy&pasted the code of
> > kset_create_and_add()/kset_create() to have a way to tell my own
> > struct kobj_type that contains the implemented my own ".child_ns_type"
> > callback.
>
> Ick.
>
> > I am writing this mail to get some advice on what I can do here
> > without doing the copy&pasted code?
> > Add a parameter to kset_create_and_add() that allows me to set an
> > "struct kobj_type"?
> > Introducing a new function that does allow me to set the new parameter
> > (I probably like that because then I don't need to update all other
> > users)?
>
> A new function would be ok, but I hate it how filesystems have to use

I will try this out. Thank you.

> "raw" kobjects and the like all the time.  It's rough, sorry.
>

DLM "Distributed Lock Manager" is a networking subsystem but as there
are mostly in-kernel filesystem users (gfs2, ocfs2) it is currently
part of "fs/".
The plan is to separate DLM lockspaces per net-namespace as each
separation acts like a "network entity" (node) that is required anyway
for each DLM socket handling.

> Also, I didn't think sysfs namespaces worked for anything except
> networking stuff.  Are you sure you really need this?  Where is your
> "namespace" defined at?
>

It is networking but I will not lie that at the end users of DLM e.g.
the cluster file system gfs2 is required to separate their sysfs
directly the same way as DLM is doing here as each mount is on a per
"network entity" (cluster node) basis.

It will be an experimental support in the first place to use it for
testing/developing (not required to synchronize traces over the
network) and scale testing (testing on a limited resource machine a
lockspace with 1000 of nodes instead of having 1000 VMs). The scale
part is important for us to make DLM ready to handle a lot of node
members to handle future use-cases.

I hope that this "justify" here the use of net namespaces in udev.

- Alex

Re: udev kset_create_and_add() with own struct kobj_type?

[Lk Sii]

On 2024/7/15 23:27, Alexander Aring wrote:
> Hi,
> 
> I am currently implementing namespace support into the fs/dlm
> subsystem. The DLM subsystem uses kset_create_and_add("dlm",
> &dlm_uevent_ops, kernel_kobj); to create the "dlm" sysfs directory.
> Underneath it creates for each lockspace a subdirectory using
> kobject_init_and_add() with a non-static name and being known during
> runtime.
> 
> Now I want to add namespace support and need to change the "default"
> kset->kobj.ktype = &kset_ktype; that is set by
> kset_create_and_add()->kset_create() to my own kobj_type because I
> need to implement a different ".child_ns_type" callback before
> kset_register() is called.
> 
you maybe use kset_register() instead of kset_create_and_add().
you maybe find example codes from below kernel files:
fs/nfs/sysfs.c
fs/f2fs/sysfs.c
fs/ubifs/sysfs.c

not sure if this info is helpful for your requirements.

> So kset_create_and_add() does not allow me to add my own
> ".child_ns_type" callback for the kset that is required for me to have
> my sysfs "kernel/dlm" directory separated by each net-namespace
> without breaking any backwards compatibility.
> 
> My current solution is that I mostly copy&pasted the code of
> kset_create_and_add()/kset_create() to have a way to tell my own
> struct kobj_type that contains the implemented my own ".child_ns_type"
> callback.
> 
> I am writing this mail to get some advice on what I can do here
> without doing the copy&pasted code?
> Add a parameter to kset_create_and_add() that allows me to set an
> "struct kobj_type"?
> Introducing a new function that does allow me to set the new parameter
> (I probably like that because then I don't need to update all other
> users)?
> 
> Thanks in advance for any advice.
> 
> - Alex
> 
> 

Re: udev kset_create_and_add() with own struct kobj_type?

[Alexander Aring]

Hi,

On Tue, Jul 16, 2024 at 8:15 AM Lk Sii <lk_sii@163.com> wrote:
>
> On 2024/7/15 23:27, Alexander Aring wrote:
> > Hi,
> >
> > I am currently implementing namespace support into the fs/dlm
> > subsystem. The DLM subsystem uses kset_create_and_add("dlm",
> > &dlm_uevent_ops, kernel_kobj); to create the "dlm" sysfs directory.
> > Underneath it creates for each lockspace a subdirectory using
> > kobject_init_and_add() with a non-static name and being known during
> > runtime.
> >
> > Now I want to add namespace support and need to change the "default"
> > kset->kobj.ktype = &kset_ktype; that is set by
> > kset_create_and_add()->kset_create() to my own kobj_type because I
> > need to implement a different ".child_ns_type" callback before
> > kset_register() is called.
> >
> you maybe use kset_register() instead of kset_create_and_add().
> you maybe find example codes from below kernel files:
> fs/nfs/sysfs.c
> fs/f2fs/sysfs.c
> fs/ubifs/sysfs.c

looked into the first one, it looks like potential new users for such
a new helper. This is what I am currently doing and it works, but I
mostly copy&paste the code of kset_create_and_add() just to set a
different "kobj.ktype" in the kset and feeling bad about it.

Thanks.

- Alex