ConfigFS + Target Mode Engine API discussion

ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Wed, 2008-09-10 at 16:31 -0700, Joel Becker wrote:
> On Wed, Sep 10, 2008 at 03:59:07PM -0700, Nicholas A. Bellinger wrote:
> > Whoops, forgot to CC you boss. :-)
> 
> 	FYI, I'll be at both KSummit and Plumbers.
> 
> Joel
> 

Great, I look forward to catching up.. :-)

Btw, I started working on the target related configfs this past weekend,
and stuck on one particular item wrt the API of a target mode fabric
plugin with a configfs enabled target engine under
$CONFIGFS/target/$FABRIC

I am making a default group under $CONFIGFS/target/core that will be
used for the generic target engine for storage object registration and
device object tuning related to target mode export, etc.  From there, I
expect the target fabric modules that are being loaded to call the
following API routines at modprobe time from a previously allocated
struct configfs_subsystem:

extern struct config_group *target_fabric_configfs_init (void)
{
        struct configfs_subsystem *subsys;

        subsys = &target_core_subsystem[0];
        return(&subsys->su_group);
}

extern struct config_group *target_fabric_configfs_register (
        struct config_group *su_cg,
        char *name,
        struct config_item_type *fabric_cit)
{

	if (!(fabric_cg = kzalloc(struct config_group, GFP_NORMAL))) {
                printk(KERN_ERR "Unable to allocate fabric_cg\n");
                ERR_PTR(-ENOMEM);
        }

        config_group_init_type_name(fabric_cg, name, fabric_cit);

}

So I am thinking about the following question:  What would be the
preferred method for calling ct_group_ops->make_group() in order to
create the $CONFIGFS/target/$FABRIC struct config_item directly from
target_fabric_configfs_register() call?  From there, the config group
hanging off $CONFIGFS/target/$FABRIC will be fabric dependent and
providing their own groups, items, depends, from the passed *fabric_cit.
How do I "simulate" a mkdir(2) configfs -> make_group() call coming from
the fabric module itself..?   This would be assuming that both mkdir(2)
and rmdir(2) would be *NOT* be able to be called directly on
$CONFIGFS/target/$FABRIC, and only the rmmod $FABRIC_MOD will call
target_fabric_configfs_unregister() and -> drop_item() to remove the
$FABRIC plugin from $CONFIGFS/target/

The configfs enabled generic target engine will need similar
functionality to simulate mkdir(2) -> make_group() and rmdir(2) ->
drop_item() in order for autoregistration of storage objects using echo
$STORAGE_OBJECT > $CONFIGFS/target/$FABRIC/$FABRIC_ENDPOINT/lun_0 to
setup the mapping as I envision.

Thanks Joel!

--nab

Re: ConfigFS + Target Mode Engine API discussion

[Joel Becker]

On Wed, Sep 10, 2008 at 06:42:46PM -0700, Nicholas A. Bellinger wrote:
> So I am thinking about the following question:  What would be the
> preferred method for calling ct_group_ops->make_group() in order to
> create the $CONFIGFS/target/$FABRIC struct config_item directly from
> target_fabric_configfs_register() call?  From there, the config group
> hanging off $CONFIGFS/target/$FABRIC will be fabric dependent and
> providing their own groups, items, depends, from the passed *fabric_cit.
> How do I "simulate" a mkdir(2) configfs -> make_group() call coming from
> the fabric module itself..?   This would be assuming that both mkdir(2)

	That's precisely what you don't do with configfs.  It's a
defined "not to be done" thing.  So there's no preferred way, there's no
way at all.
	What you want do to is drive this from mkdir().  The
make_group() will look up the sub-module it needs and return the
appropriate item.

Joel


-- 

"The lawgiver, of all beings, most owes the law allegiance.  He of all
 men should behave as though the law compelled him.  But it is the
 universal weakness of mankind that what we are given to administer we
 presently imagine we own."
        - H.G. Wells

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Wed, 2008-09-10 at 19:13 -0700, Joel Becker wrote:
> On Wed, Sep 10, 2008 at 06:42:46PM -0700, Nicholas A. Bellinger wrote:
> > So I am thinking about the following question:  What would be the
> > preferred method for calling ct_group_ops->make_group() in order to
> > create the $CONFIGFS/target/$FABRIC struct config_item directly from
> > target_fabric_configfs_register() call?  From there, the config group
> > hanging off $CONFIGFS/target/$FABRIC will be fabric dependent and
> > providing their own groups, items, depends, from the passed *fabric_cit.
> > How do I "simulate" a mkdir(2) configfs -> make_group() call coming from
> > the fabric module itself..?   This would be assuming that both mkdir(2)
> 
> 	That's precisely what you don't do with configfs.  It's a
> defined "not to be done" thing.  So there's no preferred way, there's no
> way at all.
> 	What you want do to is drive this from mkdir().  The
> make_group() will look up the sub-module it needs and return the
> appropriate item.
> 

Whew, good thing I asked about this case first..  :-)

Thanks!

--nab

> Joel
> 
> 

Re: ConfigFS + Target Mode Engine API discussion

[Joel Becker]

On Wed, Sep 10, 2008 at 09:29:18PM -0700, Nicholas A. Bellinger wrote:
> On Wed, 2008-09-10 at 19:13 -0700, Joel Becker wrote:
> > On Wed, Sep 10, 2008 at 06:42:46PM -0700, Nicholas A. Bellinger wrote:
> > > So I am thinking about the following question:  What would be the
> > > preferred method for calling ct_group_ops->make_group() in order to
> > > create the $CONFIGFS/target/$FABRIC struct config_item directly from
> > > target_fabric_configfs_register() call?  From there, the config group
> > > hanging off $CONFIGFS/target/$FABRIC will be fabric dependent and
> > > providing their own groups, items, depends, from the passed *fabric_cit.
> > > How do I "simulate" a mkdir(2) configfs -> make_group() call coming from
> > > the fabric module itself..?   This would be assuming that both mkdir(2)
> > 
> > 	That's precisely what you don't do with configfs.  It's a
> > defined "not to be done" thing.  So there's no preferred way, there's no
> > way at all.
> > 	What you want do to is drive this from mkdir().  The
> > make_group() will look up the sub-module it needs and return the
> > appropriate item.
> > 
> 
> Whew, good thing I asked about this case first..  :-)

	Can you give me a more complete description of what you're
trying to do?  that way I can maybe help with some suggestions.

Joel

-- 

"Win95 file and print sharing are for relatively friendly nets."
	- Paul Leach, Microsoft

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Wed, 2008-09-10 at 23:38 -0700, Joel Becker wrote:
> On Wed, Sep 10, 2008 at 09:29:18PM -0700, Nicholas A. Bellinger wrote:
> > On Wed, 2008-09-10 at 19:13 -0700, Joel Becker wrote:
> > > On Wed, Sep 10, 2008 at 06:42:46PM -0700, Nicholas A. Bellinger wrote:
> > > > So I am thinking about the following question:  What would be the
> > > > preferred method for calling ct_group_ops->make_group() in order to
> > > > create the $CONFIGFS/target/$FABRIC struct config_item directly from
> > > > target_fabric_configfs_register() call?  From there, the config group
> > > > hanging off $CONFIGFS/target/$FABRIC will be fabric dependent and
> > > > providing their own groups, items, depends, from the passed *fabric_cit.
> > > > How do I "simulate" a mkdir(2) configfs -> make_group() call coming from
> > > > the fabric module itself..?   This would be assuming that both mkdir(2)
> > > 
> > > 	That's precisely what you don't do with configfs.  It's a
> > > defined "not to be done" thing.  So there's no preferred way, there's no
> > > way at all.
> > > 	What you want do to is drive this from mkdir().  The
> > > make_group() will look up the sub-module it needs and return the
> > > appropriate item.
> > > 
> > 
> > Whew, good thing I asked about this case first..  :-)
> 
> 	Can you give me a more complete description of what you're
> trying to do?  that way I can maybe help with some suggestions.
> 

Sure, the process to get the things up and running would look like:

# Load in the generic target's configfs infrastructure
modprobe target_core_configfs

*) $CONFIGFS/target is created with configfs_register_subsystem()

*) $CONFIGFS/target/core is created as a default group under struct
configfs_subsystem->su_group to interact with generic target mode
engine's storage objects (eg: SCSI HCTL referenced devices, LVM UUID, MD
UUID). 

...........

 # Load LIO-Target using function symbols from target_core_configfs
modprobe iscsi_target_mod

module_init() from iscsi_target_mod (eg: the fabric module) calls
target_fabric_configfs_register() in target_core_configfs.ko, which is a
small wrapper for calling sys_mkdir($CONFIGFS/target/$FABRIC) to kick
off the struct config_group_operations->make_group() to create a struct
config_item for /sys/kernel/config/target/iscsi (or whatever the fabric
is called).

config_item_get() is also called in target_fabric_configfs_register()
and returns the newly allocated struct config_item to
iscsi_target_mod.ko code to then create new struct config_groups for
iSCSI target fabric specific abstractions and LUN mappings between
$CONFIGFS/target/core/$STORAGE_OBJECT and Fabric dependent code. I think
the latter would be done with symlinks between $FABRIC <-> Generic
Target Core Storage Object.

Anyways, knowing that make_group() and drop_item() will *NEVER* be
called internally clears up alot for me.  I will keep working on this
setup and let you know if I run into any more head scratchers.

Thanks again Joel!

--nab

Re: ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Thu, 2008-09-11 at 01:58 -0700, Nicholas A. Bellinger wrote:
> On Wed, 2008-09-10 at 23:38 -0700, Joel Becker wrote:
> > On Wed, Sep 10, 2008 at 09:29:18PM -0700, Nicholas A. Bellinger wrote:
> > > On Wed, 2008-09-10 at 19:13 -0700, Joel Becker wrote:
> > > > On Wed, Sep 10, 2008 at 06:42:46PM -0700, Nicholas A. Bellinger wrote:
> > > > > So I am thinking about the following question:  What would be the
> > > > > preferred method for calling ct_group_ops->make_group() in order to
> > > > > create the $CONFIGFS/target/$FABRIC struct config_item directly from
> > > > > target_fabric_configfs_register() call?  From there, the config group
> > > > > hanging off $CONFIGFS/target/$FABRIC will be fabric dependent and
> > > > > providing their own groups, items, depends, from the passed *fabric_cit.
> > > > > How do I "simulate" a mkdir(2) configfs -> make_group() call coming from
> > > > > the fabric module itself..?   This would be assuming that both mkdir(2)
> > > > 
> > > > 	That's precisely what you don't do with configfs.  It's a
> > > > defined "not to be done" thing.  So there's no preferred way, there's no
> > > > way at all.
> > > > 	What you want do to is drive this from mkdir().  The
> > > > make_group() will look up the sub-module it needs and return the
> > > > appropriate item.
> > > > 
> > > 
> > > Whew, good thing I asked about this case first..  :-)
> > 
> > 	Can you give me a more complete description of what you're
> > trying to do?  that way I can maybe help with some suggestions.
> > 
> 
> Sure, the process to get the things up and running would look like:
> 
> # Load in the generic target's configfs infrastructure
> modprobe target_core_configfs
> 
> *) $CONFIGFS/target is created with configfs_register_subsystem()
> 
> *) $CONFIGFS/target/core is created as a default group under struct
> configfs_subsystem->su_group to interact with generic target mode
> engine's storage objects (eg: SCSI HCTL referenced devices, LVM UUID, MD
> UUID). 
> 
> ...........
> 
>  # Load LIO-Target using function symbols from target_core_configfs
> modprobe iscsi_target_mod
> 
> module_init() from iscsi_target_mod (eg: the fabric module) calls
> target_fabric_configfs_register() in target_core_configfs.ko, which is a
> small wrapper for calling sys_mkdir($CONFIGFS/target/$FABRIC) to kick
> off the struct config_group_operations->make_group() to create a struct
> config_item for /sys/kernel/config/target/iscsi (or whatever the fabric
> is called).
> 
> config_item_get() is also called in target_fabric_configfs_register()
> and returns the newly allocated struct config_item to
> iscsi_target_mod.ko code to then create new struct config_groups for
> iSCSI target fabric specific abstractions and LUN mappings between
> $CONFIGFS/target/core/$STORAGE_OBJECT and Fabric dependent code. I think
> the latter would be done with symlinks between $FABRIC <-> Generic
> Target Core Storage Object.
> 
> Anyways, knowing that make_group() and drop_item() will *NEVER* be
> called internally clears up alot for me.  I will keep working on this
> setup and let you know if I run into any more head scratchers.
> 
> Thanks again Joel!
> 

Hi Joel,

After some more thought and a few hours of effort, I was able to get
some code up using the model above between a new target_core_configfs.ko
and iscsi_target_mod.ko (LIO-Target):

http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=commit;h=1a48fb089e1ee1cff5ef02b401d365fdf206cf69

The one symbol that I required out of fs/namei.c for vfs_mkdir() that
was not available was lookup_hash(), and the symbol is exported in the
commit so that
drivers/lio-core/transport_core_configfs.c:do_configfs_rmdir() works as
expected.  There is another version of do_configfs_rmdir() that uses the
exported lookup_one_len(), but that one is not working just quite yet.
Perhaps there is value in lookup_hash() being defined as
EXPORT_SYMBOL_GPL().

Now that the basic configfs structure between a generic engine <->
fabric is in place, the next steps will be to enable the LIO-Target  and
LIO-Core IOCTLs into make_group() and drop_item() calls.

Have a look and let me know what you think..  I will ping you when the
first pieces of LIO-Target functionality are up and running..

:-)

--nab

Re: ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Fri, 2008-09-12 at 09:24 -0700, Nicholas A. Bellinger wrote:
> On Thu, 2008-09-11 at 01:58 -0700, Nicholas A. Bellinger wrote:
> > On Wed, 2008-09-10 at 23:38 -0700, Joel Becker wrote:
> > > On Wed, Sep 10, 2008 at 09:29:18PM -0700, Nicholas A. Bellinger wrote:
> > > > On Wed, 2008-09-10 at 19:13 -0700, Joel Becker wrote:
> > > > > On Wed, Sep 10, 2008 at 06:42:46PM -0700, Nicholas A. Bellinger wrote:
> > > > > > So I am thinking about the following question:  What would be the
> > > > > > preferred method for calling ct_group_ops->make_group() in order to
> > > > > > create the $CONFIGFS/target/$FABRIC struct config_item directly from
> > > > > > target_fabric_configfs_register() call?  From there, the config group
> > > > > > hanging off $CONFIGFS/target/$FABRIC will be fabric dependent and
> > > > > > providing their own groups, items, depends, from the passed *fabric_cit.
> > > > > > How do I "simulate" a mkdir(2) configfs -> make_group() call coming from
> > > > > > the fabric module itself..?   This would be assuming that both mkdir(2)
> > > > > 
> > > > > 	That's precisely what you don't do with configfs.  It's a
> > > > > defined "not to be done" thing.  So there's no preferred way, there's no
> > > > > way at all.
> > > > > 	What you want do to is drive this from mkdir().  The
> > > > > make_group() will look up the sub-module it needs and return the
> > > > > appropriate item.
> > > > > 
> > > > 
> > > > Whew, good thing I asked about this case first..  :-)
> > > 
> > > 	Can you give me a more complete description of what you're
> > > trying to do?  that way I can maybe help with some suggestions.
> > > 
> > 
> > Sure, the process to get the things up and running would look like:
> > 
> > # Load in the generic target's configfs infrastructure
> > modprobe target_core_configfs
> > 
> > *) $CONFIGFS/target is created with configfs_register_subsystem()
> > 
> > *) $CONFIGFS/target/core is created as a default group under struct
> > configfs_subsystem->su_group to interact with generic target mode
> > engine's storage objects (eg: SCSI HCTL referenced devices, LVM UUID, MD
> > UUID). 
> > 
> > ...........
> > 
> >  # Load LIO-Target using function symbols from target_core_configfs
> > modprobe iscsi_target_mod
> > 
> > module_init() from iscsi_target_mod (eg: the fabric module) calls
> > target_fabric_configfs_register() in target_core_configfs.ko, which is a
> > small wrapper for calling sys_mkdir($CONFIGFS/target/$FABRIC) to kick
> > off the struct config_group_operations->make_group() to create a struct
> > config_item for /sys/kernel/config/target/iscsi (or whatever the fabric
> > is called).
> > 
> > config_item_get() is also called in target_fabric_configfs_register()
> > and returns the newly allocated struct config_item to
> > iscsi_target_mod.ko code to then create new struct config_groups for
> > iSCSI target fabric specific abstractions and LUN mappings between
> > $CONFIGFS/target/core/$STORAGE_OBJECT and Fabric dependent code. I think
> > the latter would be done with symlinks between $FABRIC <-> Generic
> > Target Core Storage Object.
> > 
> > Anyways, knowing that make_group() and drop_item() will *NEVER* be
> > called internally clears up alot for me.  I will keep working on this
> > setup and let you know if I run into any more head scratchers.
> > 
> > Thanks again Joel!
> > 
> 
> Hi Joel,
> 
> After some more thought and a few hours of effort, I was able to get
> some code up using the model above between a new target_core_configfs.ko
> and iscsi_target_mod.ko (LIO-Target):
> 
> http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=commit;h=1a48fb089e1ee1cff5ef02b401d365fdf206cf69
> 
> The one symbol that I required out of fs/namei.c for vfs_mkdir() that

This should have been vfs_rmdir()..  The commit requires no unexported
functions for vfs_mkdir().

--nab

> was not available was lookup_hash(), and the symbol is exported in the
> commit so that
> drivers/lio-core/transport_core_configfs.c:do_configfs_rmdir() works as
> expected.  There is another version of do_configfs_rmdir() that uses the
> exported lookup_one_len(), but that one is not working just quite yet.
> Perhaps there is value in lookup_hash() being defined as
> EXPORT_SYMBOL_GPL().
> 
> Now that the basic configfs structure between a generic engine <->
> fabric is in place, the next steps will be to enable the LIO-Target  and
> LIO-Core IOCTLs into make_group() and drop_item() calls.
> 
> Have a look and let me know what you think..  I will ping you when the
> first pieces of LIO-Target functionality are up and running..
> 
> :-)
> 
> --nab
> 
> 
> 
> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups "Linux-iSCSI.org Target Development" group.
> To post to this group, send email to linux-iscsi-target-dev@googlegroups.com
> To unsubscribe from this group, send email to linux-iscsi-target-dev+unsubscribe@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/linux-iscsi-target-dev?hl=en
> -~----------~----~----~----~------~----~------~--~---
> 

Re: ConfigFS + Target Mode Engine API discussion

[Joel Becker]

On Thu, Sep 11, 2008 at 01:58:49AM -0700, Nicholas A. Bellinger wrote:
> *) $CONFIGFS/target is created with configfs_register_subsystem()
> 
> *) $CONFIGFS/target/core is created as a default group under struct
> configfs_subsystem->su_group to interact with generic target mode
> engine's storage objects (eg: SCSI HCTL referenced devices, LVM UUID, MD
> UUID). 
> 
> ...........
> 
>  # Load LIO-Target using function symbols from target_core_configfs
> modprobe iscsi_target_mod
> 
> module_init() from iscsi_target_mod (eg: the fabric module) calls
> target_fabric_configfs_register() in target_core_configfs.ko, which is a
> small wrapper for calling sys_mkdir($CONFIGFS/target/$FABRIC) to kick
> off the struct config_group_operations->make_group() to create a struct
> config_item for /sys/kernel/config/target/iscsi (or whatever the fabric
> is called).

	No, you don't do this.  Like I said, configfs clients let
userspace create and destroy items.  Not kernelspace.  Do not call
sys_mkdir()/sys_rmdir() from your modules.
	If you don't do this, you also don't need your lookup_hash()
export.  Your code should be behaving as if it didn't know the
filesystem view existed.
	I'm trying to understand what you're actually doing.  Not the
"I want iscsi_target_mod to appear under target/iscsi" part, but why it
needs to be there in the first place.  how it all connects together.

Joel

-- 

"Every new beginning comes from some other beginning's end."

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Fri, 2008-09-12 at 13:15 -0700, Joel Becker wrote:
> On Thu, Sep 11, 2008 at 01:58:49AM -0700, Nicholas A. Bellinger wrote:
> > *) $CONFIGFS/target is created with configfs_register_subsystem()
> > 
> > *) $CONFIGFS/target/core is created as a default group under struct
> > configfs_subsystem->su_group to interact with generic target mode
> > engine's storage objects (eg: SCSI HCTL referenced devices, LVM UUID, MD
> > UUID). 
> > 
> > ...........
> > 
> >  # Load LIO-Target using function symbols from target_core_configfs
> > modprobe iscsi_target_mod
> > 
> > module_init() from iscsi_target_mod (eg: the fabric module) calls
> > target_fabric_configfs_register() in target_core_configfs.ko, which is a
> > small wrapper for calling sys_mkdir($CONFIGFS/target/$FABRIC) to kick
> > off the struct config_group_operations->make_group() to create a struct
> > config_item for /sys/kernel/config/target/iscsi (or whatever the fabric
> > is called).
> 
> 	No, you don't do this.  Like I said, configfs clients let
> userspace create and destroy items.  Not kernelspace.  Do not call
> sys_mkdir()/sys_rmdir() from your modules.
> 	If you don't do this, you also don't need your lookup_hash()
> export.  Your code should be behaving as if it didn't know the
> filesystem view existed.
> 	I'm trying to understand what you're actually doing.  Not the
> "I want iscsi_target_mod to appear under target/iscsi" part, but why it
> needs to be there in the first place.  how it all connects together.
> 

Well, the startup 'fabric registration' is only case where I figured
vfs_mkdir($CONFIGFS/target/$FABRIC) being called in order to have the
fabric struct config_item appear at modprobe $FABRIC_MOD *BEFORE* the
user did anything would be beneficial at all.   If vfs_mkdir() was not
called from transport_fabric_register_configfs(), it only means user
would have call mkdir $CONFIGFS/target/$FABRIC and/or mkdir -p
$CONFIGFS/target/$FABRIC/endpoint to kick off one common make_group()
for $FABRIC (that lives in target_core_configfs.c) and then make_group()
that likes inside of $FABRIC_MOD who's parent is $CONFIGFS/target.  In
LIO-Target's case, this would be creating a new iSCSI Qualified Name
with mkdir $CONFIGFS/target/iscsi/iqn.superturobdiskarry

The only real reason I am stuck on this is so that the modprobe $FABRIC
does not need to be followed by a user *REQUIRED* mkdir -p
$CONFIGFS/target/$FABRIC, before a normal operation like: mkdir
$CONFIGFS/target/$FABRIC/endpoint to kick off the expected two
make_groups() and the attributes in $CONFIGFS/target/$FABRIC to be
possibily tuned *BEFORE* the $FABRIC/$endpoint before the userspace
mkdir -p.  Which is not a big deal I suppose, as everything will be
there after mkdir -p $CONFIGFS/target/iscsi/iqn.superturobdiskarry
anyways, but I thought it could be useful.

Anyways, once the $FABRIC's struct config_item appears, everything from
that point is done in userspace as you (as the configfs author) would
expect with configfs structs located in $FABRIC_MOD driven by userspace
ops.  It is this method of automatic ops at modprobe $FABRIC_MOD for
struct config_item's particular struct target_fabric_configfs hanging of
the generic target's struct configfs_subsystem where any possible
benefit for automation of setup for a configfs enabled Generic Target
Engine many lie.  Same with the shutdown case for $FABRIC_MOD and
calling transport_fabric_deregister_configfs() at rmmod $FABRIC_MID.  If
vfs_rmdir() was NOT called, the user would be expected to call rmdir
$CONFIGFS/target/$FABRIC before rmmod $FABRIC_MOD.  It could be thought
of as:

A special case where vfs_mkdir() or vfs_rmdir() being called in order to
create the "multi-module" configfs subsystem containing struct
config_item_types() and struct config_*_operations in different kernel
modules..  As I don't believe the examples in
Documentation/filesystems/configfs or working code in
fs/ocfs2/cluster/nodemanager.c have this requirement AFAICT, it could be
considered a new use for configfs... :-)

I will be working on filling in target-ctl IOCTL parts in
iscsi_target_configfs.c, which look like expected configfs (not the
special case above) until you get to that other special case mentioned
earlier:  Wanting to creat a LUN mapping from a storage object from a
T10 EVPD Unit Serial to a registered
$CONFIGFS/target/$FABRIC/endpoint/lun when the
$CONFIGFS/target/core/$T10_EVPD_UNIT_SERIAL does not exist..

Thanks Joel!

--nab



> Joel
> 

Re: ConfigFS + Target Mode Engine API discussion

[Joel Becker]

On Fri, Sep 12, 2008 at 03:27:57PM -0700, Nicholas A. Bellinger wrote:
> On Fri, 2008-09-12 at 13:15 -0700, Joel Becker wrote:
> > 	No, you don't do this.  Like I said, configfs clients let
> > userspace create and destroy items.  Not kernelspace.  Do not call
> > sys_mkdir()/sys_rmdir() from your modules.
> 
> Well, the startup 'fabric registration' is only case where I figured
> vfs_mkdir($CONFIGFS/target/$FABRIC) being called in order to have the
> fabric struct config_item appear at modprobe $FABRIC_MOD *BEFORE* the
> user did anything would be beneficial at all.   If vfs_mkdir() was not
> called from transport_fabric_register_configfs(), it only means user
> would have call mkdir $CONFIGFS/target/$FABRIC and/or mkdir -p
> $CONFIGFS/target/$FABRIC/endpoint to kick off one common make_group()
> for $FABRIC (that lives in target_core_configfs.c) and then make_group()
> that likes inside of $FABRIC_MOD who's parent is $CONFIGFS/target.  In
> LIO-Target's case, this would be creating a new iSCSI Qualified Name
> with mkdir $CONFIGFS/target/iscsi/iqn.superturobdiskarry

	I'm still totally unclear as to why $FABRIC has to live under
target/.  I'm not clear if there can be more than one $FABRIC at a time.
etc.  That's what I'm trying to understand so I can help you out.

Joel

-- 

Life's Little Instruction Book #20

	"Be forgiving of yourself and others."

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Fri, 2008-09-12 at 21:49 -0700, Joel Becker wrote:
> On Fri, Sep 12, 2008 at 03:27:57PM -0700, Nicholas A. Bellinger wrote:
> > On Fri, 2008-09-12 at 13:15 -0700, Joel Becker wrote:
> > > 	No, you don't do this.  Like I said, configfs clients let
> > > userspace create and destroy items.  Not kernelspace.  Do not call
> > > sys_mkdir()/sys_rmdir() from your modules.
> > 
> > Well, the startup 'fabric registration' is only case where I figured
> > vfs_mkdir($CONFIGFS/target/$FABRIC) being called in order to have the
> > fabric struct config_item appear at modprobe $FABRIC_MOD *BEFORE* the
> > user did anything would be beneficial at all.   If vfs_mkdir() was not
> > called from transport_fabric_register_configfs(), it only means user
> > would have call mkdir $CONFIGFS/target/$FABRIC and/or mkdir -p
> > $CONFIGFS/target/$FABRIC/endpoint to kick off one common make_group()
> > for $FABRIC (that lives in target_core_configfs.c) and then make_group()
> > that likes inside of $FABRIC_MOD who's parent is $CONFIGFS/target.  In
> > LIO-Target's case, this would be creating a new iSCSI Qualified Name
> > with mkdir $CONFIGFS/target/iscsi/iqn.superturobdiskarry
> 
> 	I'm still totally unclear as to why $FABRIC has to live under
> target/.

Well, I figured that since $FABRIC would be depending upon LUN mappings
to target/core/$STORAGE_OBJECT struct config_items , it would make sense
sense for $FABRIC to appear under a common target mode engine..

Having $CONFIGFS/target/core for storage objects and
$CONFIG/$FABRIC/endpoint/lun_0 and setting up symlinks between the two
entities is what I think you have in mind.  This setup you mention may
have other advantages/disadvantages when it goes to a generic target
mode, but I have been focused on getting the initial setup (see below..)
up and running, sooo.. :-) Obviously if things can be done easier I am
all for it.  Also I think for debugging and [PROC,IOCTL] -> ConfigFS
conversion purposes, being able to call my do_configfs_[mkdir,rmdir]
wrappers when existing legacy target functionality get called to
add/remove the ConfigFS entries may have some value as well..  At least
to make it easier for kernel level $FABRIC_MOD maintainers to see how
generic kernel target mode configfs works... Just wanted to mention
that..

>   I'm not clear if there can be more than one $FABRIC at a time.

Absoulutely.  Current each $FABRIC_MOD that registers itself with the
common target engine calls the target_fabric_configfs_register() that
creates its own struct config_group with fabric dependent configfs
ops/abstractions.  This could very well be hidden behind the actual non
configfs related target engine infrastructure when $FABRIC_MOD does
actual fabric registration.  Also having common code for $FABRIC_MOD to
use to allow those LUN mappings across multiple independent $FABRIC_MODs
from target/core/$STORAGE_OBJECT is the main idea, but point taken that
$FABRIC does not necessary need to be hanging directly off
$CONFIGS/target to function..

Also why I think having $FABRIC under $CONFIGFS/target/ makes sense, I
would like to be able to do rm -rf $CONFIGFS/target to shutdown all LUN
mappings on all fabrics on all storage objects instead of rm -rf
$CONFIGFS/iscsi ; rm -rf $CONFIGFS/sas ; rm -rf $CONFIGFS/pscsi ; rm -rf
$CONFIGFS/target.. :-)

> etc.  That's what I'm trying to understand so I can help you out.
> 

:-)

So, as of this morning I have basic iqn.foo/tpgt_1 ConfigFS
functionality for LIO-Target.. Here is what it looks like..

<snip>

# Create some 

target:/sys/kernel/config/target/iscsi# mkdir -p iqn.iscsi-hd.renderbox/tpgt_1
target:/sys/kernel/config/target/iscsi# mkdir -p iqn.lio-production/tpgt_1
target:/sys/kernel/config/target/iscsi# mkdir -p iqn.upstreamtargetmodestorage/tpgt_1
target:/sys/kernel/config/target/iscsi# mkdir -p iqn.superturbodiskarray/tpgt_1
target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
target:/sys/kernel/config/target/iscsi# tree
.
|-- iqn.2003-01.org.linux-iscsi.target.i686:sn.11c045e90b
|   `-- tpgt_1
|-- iqn.2003-01.org.linux-iscsi.target.i686:sn.acc810f5a2fc
|   `-- tpgt_1
|-- iqn.2003-01.org.linux-iscsi.target.i686:sn.b341232f3595
|   `-- tpgt_1
|-- iqn.2003-01.org.linux-iscsi.target.i686:sn.b5ada4ff123a
|   `-- tpgt_1
|-- iqn.iscsi-hd.renderbox
|   `-- tpgt_1
|-- iqn.lio-production
|   `-- tpgt_1
|-- iqn.superturbodiskarray
|   `-- tpgt_1
|-- iqn.upstreamtargetmodestorage
|   `-- tpgt_1
`-- lio_version

16 directories, 1 file

# Notice the module count..
target:/sys/kernel/config/target/iscsi# lsmod
Module                  Size  Used by
iscsi_target_mod      369004  17 
target_core_configfs    11416  18 iscsi_target_mod
configfs               24216  3 iscsi_target_mod,target_core_configfs

# Delete the IQNs and TPGTs created with /sbin/iscsi-name prefix.. 
target:/sys/kernel/config/target/iscsi# rm -rf iqn.2003-01.org.linux-iscsi.target.i686\:sn.*

target:/sys/kernel/config/target/iscsi# tree
.
|-- iqn.iscsi-hd.renderbox
|   `-- tpgt_1
|-- iqn.lio-production
|   `-- tpgt_1
|-- iqn.superturbodiskarray
|   `-- tpgt_1
|-- iqn.upstreamtargetmodestorage
|   `-- tpgt_1
`-- lio_version

8 directories, 1 file

# And the module counts..
target:/sys/kernel/config/target/iscsi# lsmod
Module                  Size  Used by
iscsi_target_mod      369004  9 
target_core_configfs    11416  10 iscsi_target_mod
configfs               24216  3 iscsi_target_mod,target_core_configfs

--nab

:-)

> Joel
> 

Re: ConfigFS + Target Mode Engine API discussion

[Nicholas A. Bellinger]

On Sat, 2008-09-13 at 12:22 -0700, Nicholas A. Bellinger wrote:
> On Fri, 2008-09-12 at 21:49 -0700, Joel Becker wrote:
> > On Fri, Sep 12, 2008 at 03:27:57PM -0700, Nicholas A. Bellinger wrote:
> > > On Fri, 2008-09-12 at 13:15 -0700, Joel Becker wrote:
> > > > 	No, you don't do this.  Like I said, configfs clients let
> > > > userspace create and destroy items.  Not kernelspace.  Do not call
> > > > sys_mkdir()/sys_rmdir() from your modules.
> > > 
> > > Well, the startup 'fabric registration' is only case where I figured
> > > vfs_mkdir($CONFIGFS/target/$FABRIC) being called in order to have the
> > > fabric struct config_item appear at modprobe $FABRIC_MOD *BEFORE* the
> > > user did anything would be beneficial at all.   If vfs_mkdir() was not
> > > called from transport_fabric_register_configfs(), it only means user
> > > would have call mkdir $CONFIGFS/target/$FABRIC and/or mkdir -p
> > > $CONFIGFS/target/$FABRIC/endpoint to kick off one common make_group()
> > > for $FABRIC (that lives in target_core_configfs.c) and then make_group()
> > > that likes inside of $FABRIC_MOD who's parent is $CONFIGFS/target.  In
> > > LIO-Target's case, this would be creating a new iSCSI Qualified Name
> > > with mkdir $CONFIGFS/target/iscsi/iqn.superturobdiskarry
> > 
> > 	I'm still totally unclear as to why $FABRIC has to live under
> > target/.
> 
> Well, I figured that since $FABRIC would be depending upon LUN mappings
> to target/core/$STORAGE_OBJECT struct config_items , it would make sense
> sense for $FABRIC to appear under a common target mode engine..
> 
> Having $CONFIGFS/target/core for storage objects and
> $CONFIG/$FABRIC/endpoint/lun_0 and setting up symlinks between the two
> entities is what I think you have in mind.  This setup you mention may
> have other advantages/disadvantages when it goes to a generic target
> mode, but I have been focused on getting the initial setup (see below..)
> up and running, sooo.. :-) Obviously if things can be done easier I am
> all for it.  Also I think for debugging and [PROC,IOCTL] -> ConfigFS
> conversion purposes, being able to call my do_configfs_[mkdir,rmdir]
> wrappers when existing legacy target functionality get called to
> add/remove the ConfigFS entries may have some value as well..  At least
> to make it easier for kernel level $FABRIC_MOD maintainers to see how
> generic kernel target mode configfs works... Just wanted to mention
> that..
> 
> >   I'm not clear if there can be more than one $FABRIC at a time.
> 
> Absoulutely.  Current each $FABRIC_MOD that registers itself with the
> common target engine calls the target_fabric_configfs_register() that
> creates its own struct config_group with fabric dependent configfs
> ops/abstractions.  This could very well be hidden behind the actual non
> configfs related target engine infrastructure when $FABRIC_MOD does
> actual fabric registration.  Also having common code for $FABRIC_MOD to
> use to allow those LUN mappings across multiple independent $FABRIC_MODs
> from target/core/$STORAGE_OBJECT is the main idea, but point taken that
> $FABRIC does not necessary need to be hanging directly off
> $CONFIGS/target to function..
> 
> Also why I think having $FABRIC under $CONFIGFS/target/ makes sense, I
> would like to be able to do rm -rf $CONFIGFS/target to shutdown all LUN
> mappings on all fabrics on all storage objects instead of rm -rf
> $CONFIGFS/iscsi ; rm -rf $CONFIGFS/sas ; rm -rf $CONFIGFS/pscsi ; rm -rf
> $CONFIGFS/target.. :-)
> 
> > etc.  That's what I'm trying to understand so I can help you out.
> > 
> 
> :-)
> 
> So, as of this morning I have basic iqn.foo/tpgt_1 ConfigFS
> functionality for LIO-Target.. Here is what it looks like..
> 
> <snip>
> 
> # Create some 
> 
> target:/sys/kernel/config/target/iscsi# mkdir -p iqn.iscsi-hd.renderbox/tpgt_1
> target:/sys/kernel/config/target/iscsi# mkdir -p iqn.lio-production/tpgt_1
> target:/sys/kernel/config/target/iscsi# mkdir -p iqn.upstreamtargetmodestorage/tpgt_1
> target:/sys/kernel/config/target/iscsi# mkdir -p iqn.superturbodiskarray/tpgt_1
> target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
> target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
> target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
> target:/sys/kernel/config/target/iscsi# mkdir -p `iscsi-name`/tpgt_1
> target:/sys/kernel/config/target/iscsi# tree
> .
> |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.11c045e90b
> |   `-- tpgt_1
> |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.acc810f5a2fc
> |   `-- tpgt_1
> |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.b341232f3595
> |   `-- tpgt_1
> |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.b5ada4ff123a
> |   `-- tpgt_1
> |-- iqn.iscsi-hd.renderbox
> |   `-- tpgt_1
> |-- iqn.lio-production
> |   `-- tpgt_1
> |-- iqn.superturbodiskarray
> |   `-- tpgt_1
> |-- iqn.upstreamtargetmodestorage
> |   `-- tpgt_1
> `-- lio_version
> 
> 16 directories, 1 file
> 
> # Notice the module count..
> target:/sys/kernel/config/target/iscsi# lsmod
> Module                  Size  Used by
> iscsi_target_mod      369004  17 
> target_core_configfs    11416  18 iscsi_target_mod
> configfs               24216  3 iscsi_target_mod,target_core_configfs
> 
> # Delete the IQNs and TPGTs created with /sbin/iscsi-name prefix.. 
> target:/sys/kernel/config/target/iscsi# rm -rf iqn.2003-01.org.linux-iscsi.target.i686\:sn.*
> 
> target:/sys/kernel/config/target/iscsi# tree
> .
> |-- iqn.iscsi-hd.renderbox
> |   `-- tpgt_1
> |-- iqn.lio-production
> |   `-- tpgt_1
> |-- iqn.superturbodiskarray
> |   `-- tpgt_1
> |-- iqn.upstreamtargetmodestorage
> |   `-- tpgt_1
> `-- lio_version
> 
> 8 directories, 1 file
> 
> # And the module counts..
> target:/sys/kernel/config/target/iscsi# lsmod
> Module                  Size  Used by
> iscsi_target_mod      369004  9 
> target_core_configfs    11416  10 iscsi_target_mod
> configfs               24216  3 iscsi_target_mod,target_core_configfs
> 

Ok, just commited first working code for ops for targetname, target
portal group and Network Portal <-> target portal group logic under
$CONFIGFS/target/iscsi/$IQN/

Here is what it looks like:

export CONFIGFS=/sys/kernel/config/
export FABRIC=/sys/kernel/config/target/iscsi

# Create four iSCSI Target Nodes with Target Portal Group Tag: 1
target:/# mkdir -p $FABRIC/`iscsi-name`/tpgt_1
target:/# mkdir -p $FABRIC/`iscsi-name`/tpgt_1
target:/# mkdir -p $FABRIC/`iscsi-name`/tpgt_1
target:/# mkdir -p $FABRIC/`iscsi-name`/tpgt_1
target:/# cd $CONFIGFS
target:/sys/kernel/config# tree
.
`-- target
    |-- iscsi
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.4b471a663865
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.60a44da1dea
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.7c988087f45c
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.ed63ac2a3da7
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   `-- lio_version
    `-- version
18 directories, 2 files

# Create some iSCSI network portals on the first two targetname+tpgt tuples
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.4b471a663865/tpgt_1/np/172.16.201.137:3260
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.4b471a663865/tpgt_1/np/172.16.201.137:3261
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.4b471a663865/tpgt_1/np/172.16.201.137:3262
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.4b471a663865/tpgt_1/np/172.16.201.137:3263
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.60a44da1dea/tpgt_1/np/172.16.201.137:3260
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.60a44da1dea/tpgt_1/np/172.16.201.137:3261
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.60a44da1dea/tpgt_1/np/172.16.201.137:3262
target:/sys/kernel/config# mkdir target/iscsi/iqn.2003-01.org.linux-iscsi.target.i686\:sn.60a44da1dea/tpgt_1/np/172.16.201.137:3263

target:/sys/kernel/config# tree
.
`-- target
    |-- iscsi
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.4b471a663865
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   |           |-- 172.16.201.137:3260
    |   |           |   `-- portal_info
    |   |           |-- 172.16.201.137:3261
    |   |           |   `-- portal_info
    |   |           |-- 172.16.201.137:3262
    |   |           |   `-- portal_info
    |   |           `-- 172.16.201.137:3263
    |   |               `-- portal_info
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.60a44da1dea
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   |           |-- 172.16.201.137:3260
    |   |           |   `-- portal_info
    |   |           |-- 172.16.201.137:3261
    |   |           |   `-- portal_info
    |   |           |-- 172.16.201.137:3262
    |   |           |   `-- portal_info
    |   |           `-- 172.16.201.137:3263
    |   |               `-- portal_info
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.7c988087f45c
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   |-- iqn.2003-01.org.linux-iscsi.target.i686:sn.ed63ac2a3da7
    |   |   `-- tpgt_1
    |   |       |-- lun
    |   |       `-- np
    |   `-- lio_version
    `-- version

26 directories, 10 files

Here is the commit:

http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=commit;h=4098f1bcbe8406b882c4e265427f54a35f3c8b40

--nab