[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Coly Li]

Now the discussion moves to kernel space, I move the email from
ocfs2-tools-devel to ocfs2-devel.
The original discussion can be found from
http://oss.oracle.com/pipermail/ocfs2-tools-devel/2009-June/001891.html

Joel Becker Wrote:
> On Sat, Jun 27, 2009 at 03:46:04AM +0800, Coly Li wrote:
>> Joel Becker Wrote:
>>> On Sat, Jun 27, 2009 at 03:00:05AM +0800, Coly Li wrote:
>> [snip]
>>>> My original purpose is to find where to initiate a lvb with 64bytes, but from
>>>> mount.ocfs2 code, I don't find anywhere to create a dlm lockspace (before create
>>>> a lock) so far.
>>> 	I'm not sure why you need an LVB in mount.ocfs2, let alone a dlm
>>> lockspace.
>>>
>> Hi Joel,
>>
>> I don't need a LVB in mount code. My purpose is to find out where (k/u space)
>> creates a LVB with 64bytes length. The 64bytes LVB makes fs/dlm kernel code
>> returns an internal logic error (which should be a busy lock error) to
>> mkfs.ocfs2 when formating a mounted-by-other-node ocfs2 volume.
> 
> 	Can't fs/dlm take multiple LVB sizes?
> 
In this case, it cannot.

During mkfs.ocfs2, libdlm gets called to create a new lockspace via ioctl
handler inside fs/dlm, which (indeed) is fs/dlm/user.c:device_create_lockspace():
404 static int device_create_lockspace(struct dlm_lspace_params *params)
... ...
412
413  error = dlm_new_lockspace(params->name, strlen(params->name),
414                   &lockspace, params->flags, DLM_USER_LVB_LEN);
... ...
431 }
DLM_USER_LVB_LEN is defined to 32.

When mounting an ocfs2 volume, ocfs2 code calls dlm_new_lockspace directly.
Which is in fs/ocfs2/stack_user.c:user_cluster_connect():
814 static int user_cluster_connect(struct ocfs2_cluster_connection *conn)
815 {
... ...
842         rc = dlm_new_lockspace(conn->cc_name, strlen(conn->cc_name),
843                                &fsdlm, DLM_LSFL_FS, DLM_LVB_LEN);
... ...
853 }
DLM_LVB_LEN is 64.

When mkfs.ocfs2 tries to operate on a mounted-already ocfs2 volume, mkfs.ocfs2
calls libdlm routin and tries to create a lockspace with 32byte lvb length. The
lvb lengh is hard coded inside fs/dlm, can not be specified in user space.

When fs/dlm code tries to compare the 32bytes lvb to the 64bytes lvb which is
created in user_cluster_connect(), fs/dlm results an internal logic error(-EINVAL).

This error is misleading, if both sides use same length lvb, the returned error
can be busy domain(-EEXIST:), which is the correct one to be expected.

After checking the code of mkfs.ocfs2, mount.ocfs2, and libdlm, I don't find any
user space solution to fix the lvb length issue.

Therefore, I am locking for the possibility to use 32bytes lvb length for user
space stack. IMHO, the modification needs a ocfs2 protocol version update (for
nodes running different lvb length with user space stack), but fortunately does
not hurt on-disk format.

-- 
Coly Li
SuSE Labs

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[David Teigland]

On Mon, Jul 06, 2009 at 08:26:39PM +0800, Coly Li wrote:
> DLM_USER_LVB_LEN is defined to 32.

> DLM_LVB_LEN is 64.

Yes, the kernel dlm api allows a variable lvb size, but the user dlm api fixes
it at 32.

Do you need to actually use a 64 byte lvb from userspace?  Or do you just need
to create the locksapce with a 64 byte lvb?  We could add a flag to work
around the later fairly easily.  Changing the dlm user/kernel interface to
copy variable size lvb's would take some significant work.

Dave

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Coly Li]

David Teigland Wrote:
> On Mon, Jul 06, 2009 at 08:26:39PM +0800, Coly Li wrote:
>> DLM_USER_LVB_LEN is defined to 32.
> 
>> DLM_LVB_LEN is 64.
> 
> Yes, the kernel dlm api allows a variable lvb size, but the user dlm api fixes
> it at 32.
> 
> Do you need to actually use a 64 byte lvb from userspace?  Or do you just need
> to create the locksapce with a 64 byte lvb?  We could add a flag to work

When mkfs.ocfs2 checks a mounted ocfs2 volume, it tries to create a lockspace.
the kernel should create a 64 bytes lvb according the existed lvb created by
kernel dlm api.
In this case, the answer might be, to allow user space utilities to create 64
bytes lvb.

> around the later fairly easily.  Changing the dlm user/kernel interface to
> copy variable size lvb's would take some significant work.
> 

I checked ocfs2-tools code, it seems libo2dlm uses 32 bytes lvb.
Is it possible just extend DLM_USER_LVB_LEN to 64 bytes ? It seems OK for
ocfs2-tools code, but I don't know whether there is other negative effect.

Thanks.

-- 
Coly Li
SuSE Labs

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Sunil Mushran]

David Teigland wrote:
> Yes, the kernel dlm api allows a variable lvb size, but the user dlm api fixes
> it at 32.
>
> Do you need to actually use a 64 byte lvb from userspace?  Or do you just need
> to create the locksapce with a 64 byte lvb?  We could add a flag to work
> around the later fairly easily.  Changing the dlm user/kernel interface to
> copy variable size lvb's would take some significant work.

I guess the problem here is that Coly is attempting to run mkfs on a volume
that is mounted on another node. mkfs.ocfs2 joins the lockspace to ensure it
is not in use across the cluster, before cleaning out the superblock. If 
it is
mounted, the lockspace would have been created by the fs... meaning 
64-byte lvb.

Coly, is this correct?

If so, then yes, the flag workaround to create a fixed 64-byte lvb will do.

Thanks
Sunil

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Joel Becker]

On Tue, Jul 07, 2009 at 11:01:13AM -0500, David Teigland wrote:
> On Mon, Jul 06, 2009 at 08:26:39PM +0800, Coly Li wrote:
> > DLM_USER_LVB_LEN is defined to 32.
> 
> > DLM_LVB_LEN is 64.
> 
> Yes, the kernel dlm api allows a variable lvb size, but the user dlm api fixes
> it at 32.
> 
> Do you need to actually use a 64 byte lvb from userspace?  Or do you just need
> to create the locksapce with a 64 byte lvb?  We could add a flag to work
> around the later fairly easily.  Changing the dlm user/kernel interface to
> copy variable size lvb's would take some significant work.

	In this case, it's userspace utilities locking to verify no one
else is doing anything in the cluster.  coly has noticed mkfs.ocfs2, but
tunefs.ocfs2 and fsck.ocfs2 do the same thing.
	While a 64byte user lvb would be ideal, I think a flag to work
around it would be great.  If the flag says "I know this lockspace may
have !32byte LVBs, but I promise not to use them", and you can error
when someone tries to set/get LVBs in that case, I think it works.

Joel

-- 

"Practice random acts of kindness and senseless acts of beauty."

 Oh, and don't forget where your towel is.

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Coly Li]

Sunil Mushran Wrote:
> David Teigland wrote:
>> Yes, the kernel dlm api allows a variable lvb size, but the user dlm
>> api fixes
>> it at 32.
>>
>> Do you need to actually use a 64 byte lvb from userspace?  Or do you
>> just need
>> to create the locksapce with a 64 byte lvb?  We could add a flag to work
>> around the later fairly easily.  Changing the dlm user/kernel
>> interface to
>> copy variable size lvb's would take some significant work.
> 
> I guess the problem here is that Coly is attempting to run mkfs on a volume
> that is mounted on another node. mkfs.ocfs2 joins the lockspace to
> ensure it
> is not in use across the cluster, before cleaning out the superblock. If
> it is
> mounted, the lockspace would have been created by the fs... meaning
> 64-byte lvb.
> 
> Coly, is this correct?

Yes, this is what I mean.



-- 
Coly Li
SuSE Labs

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[David Teigland]

On Tue, Jul 07, 2009 at 10:47:42AM -0700, Joel Becker wrote:
> On Tue, Jul 07, 2009 at 11:01:13AM -0500, David Teigland wrote:
> > On Mon, Jul 06, 2009 at 08:26:39PM +0800, Coly Li wrote:
> > > DLM_USER_LVB_LEN is defined to 32.
> > 
> > > DLM_LVB_LEN is 64.
> > 
> > Yes, the kernel dlm api allows a variable lvb size, but the user dlm api fixes
> > it at 32.
> > 
> > Do you need to actually use a 64 byte lvb from userspace?  Or do you just need
> > to create the locksapce with a 64 byte lvb?  We could add a flag to work
> > around the later fairly easily.  Changing the dlm user/kernel interface to
> > copy variable size lvb's would take some significant work.
> 
> 	In this case, it's userspace utilities locking to verify no one
> else is doing anything in the cluster.  coly has noticed mkfs.ocfs2, but
> tunefs.ocfs2 and fsck.ocfs2 do the same thing.
> 	While a 64byte user lvb would be ideal, I think a flag to work
> around it would be great.  If the flag says "I know this lockspace may
> have !32byte LVBs, but I promise not to use them", and you can error
> when someone tries to set/get LVBs in that case, I think it works.

OK, I should have a patch for you to try in the next day or two.
Dave

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[David Teigland]

On Wed, Jul 08, 2009 at 09:15:42AM -0500, David Teigland wrote:
> On Tue, Jul 07, 2009 at 10:47:42AM -0700, Joel Becker wrote:
> > On Tue, Jul 07, 2009 at 11:01:13AM -0500, David Teigland wrote:
> > > On Mon, Jul 06, 2009 at 08:26:39PM +0800, Coly Li wrote:
> > > > DLM_USER_LVB_LEN is defined to 32.
> > > 
> > > > DLM_LVB_LEN is 64.
> > > 
> > > Yes, the kernel dlm api allows a variable lvb size, but the user dlm api fixes
> > > it at 32.
> > > 
> > > Do you need to actually use a 64 byte lvb from userspace?  Or do you just need
> > > to create the locksapce with a 64 byte lvb?  We could add a flag to work
> > > around the later fairly easily.  Changing the dlm user/kernel interface to
> > > copy variable size lvb's would take some significant work.
> > 
> > 	In this case, it's userspace utilities locking to verify no one
> > else is doing anything in the cluster.  coly has noticed mkfs.ocfs2, but
> > tunefs.ocfs2 and fsck.ocfs2 do the same thing.
> > 	While a 64byte user lvb would be ideal, I think a flag to work
> > around it would be great.  If the flag says "I know this lockspace may
> > have !32byte LVBs, but I promise not to use them", and you can error
> > when someone tries to set/get LVBs in that case, I think it works.

Here's a kernel patch that I've not yet tried.  The code using libdlm will
need to add the flag 0x00000010 to dlm_new_lockspace().  (Until we've added
the new LVB64 define to libdlm.h.)

From 75e92c78bb0b0002f0253bfe10bef8585a0d52d6 Mon Sep 17 00:00:00 2001
From: David Teigland <teigland@redhat.com>
Date: Thu, 9 Jul 2009 13:11:02 -0500
Subject: [PATCH] dlm: add DLM_LSFL_LVB64 for lockspace creation

Add a new flag DLM_LSFL_LVB64 to create lockspaces with 64 byte lvbs,
overriding the lvb length parameter.  It is useful to override the
fixed 32 byte lvb imposed by the user api when the lockspace is created
from userspace but used from the kernel.  Locking through the user api
can still only access 32 byte lvbs.

Ocfs2 user tools create lockspaces that may be in use by the file
system on other nodes.  The file system creates and uses 64 byte lvbs.
Creating the lockspace from user tools fails unless the default 32 byte
lvb is overriden to match the 64 byte length already in use.

Signed-off-by: David Teigland <teigland@redhat.com>
---
 fs/dlm/lockspace.c  |    5 ++++-
 include/linux/dlm.h |   28 +++++++++++++++++++++++++---
 2 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/fs/dlm/lockspace.c b/fs/dlm/lockspace.c
index d489fcc..4d5b5d5 100644
--- a/fs/dlm/lockspace.c
+++ b/fs/dlm/lockspace.c
@@ -391,6 +391,9 @@ static int new_lockspace(const char *name, int namelen, void **lockspace,
 	int i, size, error;
 	int do_unreg = 0;
 
+	if (flags & DLM_LSFL_LVB64)
+		lvblen = 64;
+
 	if (namelen > DLM_LOCKSPACE_LEN)
 		return -EINVAL;
 
@@ -451,7 +454,7 @@ static int new_lockspace(const char *name, int namelen, void **lockspace,
 	/* ls_exflags are forced to match among nodes, and we don't
 	   need to require all nodes to have some flags set */
 	ls->ls_exflags = (flags & ~(DLM_LSFL_TIMEWARN | DLM_LSFL_FS |
-				    DLM_LSFL_NEWEXCL));
+				    DLM_LSFL_NEWEXCL | DLM_LSFL_LVB64));
 
 	size = dlm_config.ci_rsbtbl_size;
 	ls->ls_rsbtbl_size = size;
diff --git a/include/linux/dlm.h b/include/linux/dlm.h
index 0b3518c..24a62de 100644
--- a/include/linux/dlm.h
+++ b/include/linux/dlm.h
@@ -65,12 +65,34 @@ struct dlm_lksb {
 	char *	 sb_lvbptr;
 };
 
-/* dlm_new_lockspace() flags */
+/*
+ * dlm_new_lockspace() flags
+ *
+ * DLM_LSFL_NODIR: Do not use a resource directory to determine the master of
+ * resources, but set the master statically by hashing the resource name (the
+ * same hash that would otherwise determine the resource directory node).
+ *
+ * DLM_LSFL_TIMEWARN: Track how long locks are waiting to be granted and send a
+ * notice to userspace (via netlink) if the time exceeds
+ * dlm_config.ci_timewarn_cs centiseconds.
+ *
+ * DLM_LSFL_FS: Use GFP_NOFS for memory allocations instead of GFP_KERNEL.
+ * This is set by filesystems using the dlm.
+ *
+ * DLM_LSFL_NEWEXCL: Create the lockspace only if it does not already exist.
+ * If the lockspace already exists locally, -EEXIST is returned.
+ *
+ * DLM_LSFL_LVB64: The user api has no lvb length parameter, so it creates
+ * lockspaces with 32 byte lvbs by default, or 64 byte lvbs with the LVB64
+ * flag.  LVB64 is useful for lockspaces created from userspace but used from
+ * the kernel; the user api still only reads/writes 32 byte lvbs.
+ */
 
 #define DLM_LSFL_NODIR		0x00000001
 #define DLM_LSFL_TIMEWARN	0x00000002
-#define DLM_LSFL_FS     	0x00000004
-#define DLM_LSFL_NEWEXCL     	0x00000008
+#define DLM_LSFL_FS		0x00000004
+#define DLM_LSFL_NEWEXCL	0x00000008
+#define DLM_LSFL_LVB64		0x00000010
 
 #ifdef __KERNEL__
 
-- 
1.5.5.6

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Joel Becker]

On Thu, Jul 09, 2009 at 01:53:38PM -0500, David Teigland wrote:
> Here's a kernel patch that I've not yet tried.  The code using libdlm will
> need to add the flag 0x00000010 to dlm_new_lockspace().  (Until we've added
> the new LVB64 define to libdlm.h.)
> 
> >From 75e92c78bb0b0002f0253bfe10bef8585a0d52d6 Mon Sep 17 00:00:00 2001
> From: David Teigland <teigland@redhat.com>
> Date: Thu, 9 Jul 2009 13:11:02 -0500
> Subject: [PATCH] dlm: add DLM_LSFL_LVB64 for lockspace creation
> 
> Add a new flag DLM_LSFL_LVB64 to create lockspaces with 64 byte lvbs,
> overriding the lvb length parameter.  It is useful to override the
> fixed 32 byte lvb imposed by the user api when the lockspace is created
> from userspace but used from the kernel.  Locking through the user api
> can still only access 32 byte lvbs.

	Hmm, you're tying a permanent flag to 64bytes, yet userspace
can't get at them?  Seems a bit too ocfs2-tools specific, and yet it
doesn't let ocfs2-tools eventually try to share the lvb.

Joel

-- 

"But all my words come back to me
 In shades of mediocrity.
 Like emptiness in harmony
 I need someone to comfort me."

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[David Teigland]

On Thu, Jul 09, 2009 at 01:55:28PM -0700, Joel Becker wrote:
> On Thu, Jul 09, 2009 at 01:53:38PM -0500, David Teigland wrote:
> > Here's a kernel patch that I've not yet tried.  The code using libdlm will
> > need to add the flag 0x00000010 to dlm_new_lockspace().  (Until we've added
> > the new LVB64 define to libdlm.h.)
> > 
> > >From 75e92c78bb0b0002f0253bfe10bef8585a0d52d6 Mon Sep 17 00:00:00 2001
> > From: David Teigland <teigland@redhat.com>
> > Date: Thu, 9 Jul 2009 13:11:02 -0500
> > Subject: [PATCH] dlm: add DLM_LSFL_LVB64 for lockspace creation
> > 
> > Add a new flag DLM_LSFL_LVB64 to create lockspaces with 64 byte lvbs,
> > overriding the lvb length parameter.  It is useful to override the
> > fixed 32 byte lvb imposed by the user api when the lockspace is created
> > from userspace but used from the kernel.  Locking through the user api
> > can still only access 32 byte lvbs.
> 
> 	Hmm, you're tying a permanent flag to 64bytes, yet userspace
> can't get at them?  Seems a bit too ocfs2-tools specific, and yet it
> doesn't let ocfs2-tools eventually try to share the lvb.

Yes, I think the only other sane option is to do full variable-sized lvb
support for userspace.  That would be nice, and I wouldn't mind doing it.  But
it would take significant user/kernel api work, and take some time, of course.

Dave

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Joel Becker]

On Thu, Jul 09, 2009 at 04:55:04PM -0500, David Teigland wrote:
> On Thu, Jul 09, 2009 at 01:55:28PM -0700, Joel Becker wrote:
> > On Thu, Jul 09, 2009 at 01:53:38PM -0500, David Teigland wrote:
> > > Here's a kernel patch that I've not yet tried.  The code using libdlm will
> > > need to add the flag 0x00000010 to dlm_new_lockspace().  (Until we've added
> > > the new LVB64 define to libdlm.h.)
> > > 
> > > >From 75e92c78bb0b0002f0253bfe10bef8585a0d52d6 Mon Sep 17 00:00:00 2001
> > > From: David Teigland <teigland@redhat.com>
> > > Date: Thu, 9 Jul 2009 13:11:02 -0500
> > > Subject: [PATCH] dlm: add DLM_LSFL_LVB64 for lockspace creation
> > > 
> > > Add a new flag DLM_LSFL_LVB64 to create lockspaces with 64 byte lvbs,
> > > overriding the lvb length parameter.  It is useful to override the
> > > fixed 32 byte lvb imposed by the user api when the lockspace is created
> > > from userspace but used from the kernel.  Locking through the user api
> > > can still only access 32 byte lvbs.
> > 
> > 	Hmm, you're tying a permanent flag to 64bytes, yet userspace
> > can't get at them?  Seems a bit too ocfs2-tools specific, and yet it
> > doesn't let ocfs2-tools eventually try to share the lvb.
> 
> Yes, I think the only other sane option is to do full variable-sized lvb
> support for userspace.  That would be nice, and I wouldn't mind doing it.  But
> it would take significant user/kernel api work, and take some time, of course.

	Well, my original idea of an IGNORE_LVB flag, with code in
libdlm to error if the LVB functoins are used, would not be tied to a
specific LVB size and would prevent miss-matched LVB ops.

Joel

-- 

"Reader, suppose you were and idiot.  And suppose you were a member of
 Congress.  But I repeat myself."
	- Mark Twain

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Coly Li]

David Teigland Wrote:
> On Wed, Jul 08, 2009 at 09:15:42AM -0500, David Teigland wrote:
>> On Tue, Jul 07, 2009 at 10:47:42AM -0700, Joel Becker wrote:
>>> 	In this case, it's userspace utilities locking to verify no one
>>> else is doing anything in the cluster.  coly has noticed mkfs.ocfs2, but
>>> tunefs.ocfs2 and fsck.ocfs2 do the same thing.
>>> 	While a 64byte user lvb would be ideal, I think a flag to work
>>> around it would be great.  If the flag says "I know this lockspace may
>>> have !32byte LVBs, but I promise not to use them", and you can error
>>> when someone tries to set/get LVBs in that case, I think it works.
> 
> Here's a kernel patch that I've not yet tried.  The code using libdlm will
> need to add the flag 0x00000010 to dlm_new_lockspace().  (Until we've added
> the new LVB64 define to libdlm.h.)
> 
> +#define DLM_LSFL_LVB64		0x00000010
>
Another difficulty is user space libdlm interface. Right now, the interface
mkfs.ocfs2 uses to create a lock space is,
dlm_lshandle_t dlm_create_lockspace(const char *name, mode_t mode)
{
        return create_lockspace(name, mode, 0);
}

There is no way to set flags in dlm_create_lockspace() and it is set to 0 inside
the interface.

Is it possible to export create_lockspace() and release_lockspace() from libdlm
? So ocfs2-tools can load these symbols with dlopen() and calls these routines
with flags (set DLM_LSFL_LVB64).

Other 2 options might be,
1) add flags in dlm_create_lockspace(), which will change libdlm api.
2) update default lvb length from 32bytes to 64bytes.

Thanks.
-- 
Coly Li
SuSE Labs

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[David Teigland]

On Fri, Jul 31, 2009 at 07:59:08PM +0800, Coly Li wrote:
> 
> 
> David Teigland Wrote:
> > On Wed, Jul 08, 2009 at 09:15:42AM -0500, David Teigland wrote:
> >> On Tue, Jul 07, 2009 at 10:47:42AM -0700, Joel Becker wrote:
> >>> 	In this case, it's userspace utilities locking to verify no one
> >>> else is doing anything in the cluster.  coly has noticed mkfs.ocfs2, but
> >>> tunefs.ocfs2 and fsck.ocfs2 do the same thing.
> >>> 	While a 64byte user lvb would be ideal, I think a flag to work
> >>> around it would be great.  If the flag says "I know this lockspace may
> >>> have !32byte LVBs, but I promise not to use them", and you can error
> >>> when someone tries to set/get LVBs in that case, I think it works.
> > 
> > Here's a kernel patch that I've not yet tried.  The code using libdlm will
> > need to add the flag 0x00000010 to dlm_new_lockspace().  (Until we've added
> > the new LVB64 define to libdlm.h.)
> > 
> > +#define DLM_LSFL_LVB64		0x00000010
> >
> Another difficulty is user space libdlm interface. Right now, the interface
> mkfs.ocfs2 uses to create a lock space is,
> dlm_lshandle_t dlm_create_lockspace(const char *name, mode_t mode)
> {
>         return create_lockspace(name, mode, 0);
> }
> 
> There is no way to set flags in dlm_create_lockspace() and it is set to 0 inside
> the interface.
> 
> Is it possible to export create_lockspace() and release_lockspace() from libdlm
> ? So ocfs2-tools can load these symbols with dlopen() and calls these routines
> with flags (set DLM_LSFL_LVB64).
> 
> Other 2 options might be,
> 1) add flags in dlm_create_lockspace(), which will change libdlm api.
> 2) update default lvb length from 32bytes to 64bytes.

I don't think I'll bother with the LVB64 flag hack.  Instead, I'm planning to
do a larger API change to really support variable length lvb's from userspace.

It's actually dlm_new_lockspace() that you would use to pass flags, and once
we add variable length lvb's we'll need a new third call that adds an lvblen
parameter.

Dave

[Ocfs2-devel] lvb length issue [was Re: [ocfs2-tools-devel] question of ocfs2_controld (Jun 27)]

[Coly Li]

David Teigland Wrote:
> On Fri, Jul 31, 2009 at 07:59:08PM +0800, Coly Li wrote:
>>
>> David Teigland Wrote:
>>> On Wed, Jul 08, 2009 at 09:15:42AM -0500, David Teigland wrote:
>>>> On Tue, Jul 07, 2009 at 10:47:42AM -0700, Joel Becker wrote:
>>>>> 	In this case, it's userspace utilities locking to verify no one
>>>>> else is doing anything in the cluster.  coly has noticed mkfs.ocfs2, but
>>>>> tunefs.ocfs2 and fsck.ocfs2 do the same thing.
>>>>> 	While a 64byte user lvb would be ideal, I think a flag to work
>>>>> around it would be great.  If the flag says "I know this lockspace may
>>>>> have !32byte LVBs, but I promise not to use them", and you can error
>>>>> when someone tries to set/get LVBs in that case, I think it works.
>>> Here's a kernel patch that I've not yet tried.  The code using libdlm will
>>> need to add the flag 0x00000010 to dlm_new_lockspace().  (Until we've added
>>> the new LVB64 define to libdlm.h.)
>>>
>>> +#define DLM_LSFL_LVB64		0x00000010
>>>
>> Another difficulty is user space libdlm interface. Right now, the interface
>> mkfs.ocfs2 uses to create a lock space is,
>> dlm_lshandle_t dlm_create_lockspace(const char *name, mode_t mode)
>> {
>>         return create_lockspace(name, mode, 0);
>> }
>>
>> There is no way to set flags in dlm_create_lockspace() and it is set to 0 inside
>> the interface.
>>
>> Is it possible to export create_lockspace() and release_lockspace() from libdlm
>> ? So ocfs2-tools can load these symbols with dlopen() and calls these routines
>> with flags (set DLM_LSFL_LVB64).
>>
>> Other 2 options might be,
>> 1) add flags in dlm_create_lockspace(), which will change libdlm api.
>> 2) update default lvb length from 32bytes to 64bytes.
> 
> I don't think I'll bother with the LVB64 flag hack.  Instead, I'm planning to
> do a larger API change to really support variable length lvb's from userspace.
> 
> It's actually dlm_new_lockspace() that you would use to pass flags, and once
> we add variable length lvb's we'll need a new third call that adds an lvblen
> parameter.
> 

I see. Once the patches are available, I can help to test the new API.

Thanks.
-- 
Coly Li
SuSE Labs