[PATCH 0/9] Udev latency

[PATCH 0/9] Udev latency

[Zdenek Kabelac]

WARNING  - Not a final patch - this is patch for discussion WARNING

Patch addresses udev latencies we may observe with commands
like 'vgchange -ay' when the command waits after dm table is created
also for udev to process its rules for each every created device.
This adds uncessary delays for larger LV sets.

Here are some basic numbers -

First number is taken on clean tree and (the second) is my own modified
code that has few more accelerations i.e. memlock & config parsing mods
which are however not yet in generably 'usable' form.
During tests - gvfs/gdu/udisk daemons were killed.

My testing vg contains 1708 linear devices.

vgchange -ay

Without udev in use (with current todays rawhide udev-164-6 package
which is in fact noticable slower) -   1:30.6 (55.2sec)
When latency patch is in use -         1:29  (44.8sec) (10sec diff - ~23%)

vgchange -an

Without udev latency patch -  2:35 (1:33.7)
With udev latency patch -     2:34 (1:31.4) (so only 2sec difference! ~3%)

NOTE:
 Tests've been repeated several times - times are relatively stable.
 Another interesting note - before today's udev upgrade - it's been 30%/5%.
 With older udev - thing were faster, and differences were bigger.
 
TODO:
 Release memlock & config parser patches - as they give big improvements.
 Though I want to discusse them first a bit - as the proper solution could
 be a bit tricky.

Zdenek Kabelac (9):
  Add get_lib_cookie
  Move fs_unlock
  Add lvm_do_unlock_fs
  Use internal cookie for dm_tree functions
  Stop calling fs_unlock() and dm_udev_wait()
  Add  LCK_UNLOCK_FS  and unlock_fs
  Add CLVMD_CMD_UNLOCK_FS command
  Locking with unlock_fs
  Wait for devices being created

 daemons/clvmd/clvm.h          |    1 +
 daemons/clvmd/clvmd-command.c |    6 ++
 daemons/clvmd/clvmd.c         |    3 +
 daemons/clvmd/lvm-functions.c |    7 +++
 daemons/clvmd/lvm-functions.h |    1 +
 lib/activate/activate.c       |    5 ++
 lib/activate/activate.h       |    2 +
 lib/activate/dev_manager.c    |   17 +------
 lib/activate/fs.c             |    1 +
 lib/activate/fs.h             |    1 -
 lib/locking/cluster_locking.c |    7 +++
 lib/locking/file_locking.c    |    6 ++
 lib/locking/locking.h         |    4 ++
 lib/metadata/lv_manip.c       |    3 +
 libdm/libdm-common.c          |   23 +++++++++
 libdm/libdm-common.h          |    1 +
 libdm/libdm-deptree.c         |  106 +++++++++++++++++++++++++++++++++++-----
 tools/lvmcmdline.c            |    3 +
 18 files changed, 166 insertions(+), 31 deletions(-)

--
1.7.3.3

[PATCH 1/9] Add get_lib_cookie

[Zdenek Kabelac]

Create internal libdm function to access internal cookie value.
If the _lib_cookie is 0 - new value is generated.

This leads to creation of possibly unused semaphore - but currenly
it simplifies patch.

Cookie is automaticaly destroyed when   update_devs() is executed.
This currently makes imposible to use only 1 cookie per whole
executable lifetime - as  dm_udev_wait currently destroys semaphor
when counter drops to 0.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 libdm/libdm-common.c |   23 +++++++++++++++++++++++
 libdm/libdm-common.h |    1 +
 2 files changed, 24 insertions(+), 0 deletions(-)

diff --git a/libdm/libdm-common.c b/libdm/libdm-common.c
index 2dfa28f..7407163 100644
--- a/libdm/libdm-common.c
+++ b/libdm/libdm-common.c
@@ -65,6 +65,7 @@ static int _udev_running = -1;
 static int _sync_with_udev = 1;
 static int _udev_checking = 1;
 #endif
+static uint32_t _lib_cookie;
 
 /*
  * Library users can provide their own logging
@@ -811,6 +812,13 @@ int set_dev_node_read_ahead(const char *dev_name, uint32_t read_ahead,
 
 void update_devs(void)
 {
+	if (_lib_cookie) {
+		if (!dm_udev_wait(_lib_cookie))
+			stack;
+
+		_lib_cookie = 0;
+	}
+
 	_pop_node_ops();
 }
 
@@ -1295,3 +1303,18 @@ repeat_wait:
 }
 
 #endif		/* UDEV_SYNC_SUPPORT */
+
+int get_lib_cookie(uint32_t *cookie)
+{
+	if (!_lib_cookie) {
+		if (!dm_udev_get_sync_support())
+			return 0;
+
+		if (!dm_udev_create_cookie(&_lib_cookie))
+			return_0;
+	}
+
+	*cookie = _lib_cookie;
+
+	return 1;
+}
diff --git a/libdm/libdm-common.h b/libdm/libdm-common.h
index fcb334f..639e4d5 100644
--- a/libdm/libdm-common.h
+++ b/libdm/libdm-common.h
@@ -32,4 +32,5 @@ int set_dev_node_read_ahead(const char *dev_name, uint32_t read_ahead,
 			    uint32_t read_ahead_flags);
 void update_devs(void);
 
+int get_lib_cookie(uint32_t *cookie);
 #endif
-- 
1.7.3.3

[PATCH 2/9] Move fs_unlock

[Zdenek Kabelac]

Make fs_unlock() easily linkable function for clvmd code.
Move it for now to active.h -  maybe some better function name
is needed.

fs_unlock() is executing update_devs() -> hidden wait on semaphore.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 lib/activate/activate.h |    2 ++
 lib/activate/fs.c       |    1 +
 lib/activate/fs.h       |    1 -
 3 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/lib/activate/activate.h b/lib/activate/activate.h
index df46ac2..329893d 100644
--- a/lib/activate/activate.h
+++ b/lib/activate/activate.h
@@ -101,6 +101,8 @@ int lv_has_target_type(struct dm_pool *mem, struct logical_volume *lv,
 int monitor_dev_for_events(struct cmd_context *cmd, struct logical_volume *lv,
 			   unsigned origin_only, int do_reg);
 
+void fs_unlock(void);
+
 #ifdef DMEVENTD
 #  include "libdevmapper-event.h"
 char *get_monitor_dso_path(struct cmd_context *cmd, const char *libpath);
diff --git a/lib/activate/fs.c b/lib/activate/fs.c
index b78da11..a169ab0 100644
--- a/lib/activate/fs.c
+++ b/lib/activate/fs.c
@@ -15,6 +15,7 @@
 
 #include "lib.h"
 #include "fs.h"
+#include "activate.h"
 #include "toolcontext.h"
 #include "lvm-string.h"
 #include "lvm-file.h"
diff --git a/lib/activate/fs.h b/lib/activate/fs.h
index 28b2c73..640ce9a 100644
--- a/lib/activate/fs.h
+++ b/lib/activate/fs.h
@@ -29,6 +29,5 @@ int fs_del_lv_byname(const char *dev_dir, const char *vg_name,
 		     const char *lv_name, int check_udev);
 int fs_rename_lv(struct logical_volume *lv, const char *dev, 
 		 const char *old_vgname, const char *old_lvname);
-void fs_unlock(void);
 
 #endif
-- 
1.7.3.3

[PATCH 3/9] Add lvm_do_unlock_fs

[Zdenek Kabelac]

Introduce lvm_do_unlock_fs() function to call lvm's fs_unlock()
while holding lvm_lock mutex.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 daemons/clvmd/lvm-functions.c |    7 +++++++
 daemons/clvmd/lvm-functions.h |    1 +
 2 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/daemons/clvmd/lvm-functions.c b/daemons/clvmd/lvm-functions.c
index 0520ae9..187acc2 100644
--- a/daemons/clvmd/lvm-functions.c
+++ b/daemons/clvmd/lvm-functions.c
@@ -894,6 +894,13 @@ struct dm_hash_node *get_next_excl_lock(struct dm_hash_node *v, char **name)
 	return v;
 }
 
+void lvm_do_unlock_fs(void)
+{
+	pthread_mutex_lock(&lvm_lock);
+	fs_unlock();
+	pthread_mutex_unlock(&lvm_lock);
+}
+
 /* Called to initialise the LVM context of the daemon */
 int init_clvm(int using_gulm, char **argv)
 {
diff --git a/daemons/clvmd/lvm-functions.h b/daemons/clvmd/lvm-functions.h
index 97153d4..6558863 100644
--- a/daemons/clvmd/lvm-functions.h
+++ b/daemons/clvmd/lvm-functions.h
@@ -36,5 +36,6 @@ extern char *get_last_lvm_error(void);
 extern void do_lock_vg(unsigned char command, unsigned char lock_flags,
 		      char *resource);
 extern struct dm_hash_node *get_next_excl_lock(struct dm_hash_node *v, char **name);
+void lvm_do_unlock_fs(void);
 
 #endif
-- 
1.7.3.3

[PATCH 4/9] Use internal cookie for dm_tree functions

[Zdenek Kabelac]

Creative part of patchset - start to use internal cookie for:
dm_tree_deactivate_children()
dm_tree_activate_children()
dm_tree_preload_children()

Patch is not so simple (and I'm not 100% sure it covers all variants).
Tries to switch dm_tree cookie/shmid to internal one if the supplied
cookie/shmid is 0 - and it tries to remember this for preload.
It would make code easier if we would care only about the internal libdm
cookie/shmid and would not allow external dm_tree_set_cookie.
Actually is there any other user of dm_tree except for lvm?

The problem here is 'immediate' node - which resets cookie/shmid value.
So we would probably need to add another extra variable to set
whether libdm should use internal cookie/shmid numbers - code tries to
handle this case - but the best would be to drop support for external
cookie/shmid.

Note: get_lib_cookie always generate 'internal' cookie/shmid even if the user
supplies externally created cookie/shmid (or the cookie/shmid is not needed
because no device needs to be created/destroyed).

Primarily patch is able to show speedup from udev latency.

I think solution would be to hide this internal cookie/shmid in
dm_task_set_cookie() function - but I've currently decided to 'drop' this
larger code modification - as it may influence behavior of 'purely'
libdm users like Milan's dmcrypt or dmraid.

IMHO libdm should completly hide 'udev' existance and we should
provide library abstranction over udev - but that needs wider
agreement.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 libdm/libdm-deptree.c |  106 ++++++++++++++++++++++++++++++++++++++++++------
 1 files changed, 92 insertions(+), 14 deletions(-)

diff --git a/libdm/libdm-deptree.c b/libdm/libdm-deptree.c
index 8d00514..d801eec 100644
--- a/libdm/libdm-deptree.c
+++ b/libdm/libdm-deptree.c
@@ -1102,6 +1102,36 @@ static int _suspend_node(const char *name, uint32_t major, uint32_t minor,
 	return r;
 }
 
+static int _use_lib_cookie(struct dm_tree_node *dnode, uint32_t *cookie)
+{
+	uint32_t c;
+
+	if (!get_lib_cookie(&c))
+		return_0;
+
+	*cookie = dm_tree_get_cookie(dnode);
+
+	if (!*cookie) {
+		*cookie = c;
+		dm_tree_set_cookie(dnode, c);
+	} else if (*cookie != c)
+		*cookie = 0;
+
+	return 1;
+}
+
+static int _wait_on_cookie(struct dm_tree_node *dnode, uint32_t cookie)
+{
+	if (!cookie && (cookie = dm_tree_get_cookie(dnode))) {
+		if (!dm_udev_wait(cookie))
+			return_0;
+
+		dm_tree_set_cookie(dnode, 0);
+	}
+
+	return 1;
+}
+
 /*
  * FIXME Don't attempt to deactivate known internal dependencies.
  */
@@ -1185,7 +1215,18 @@ int dm_tree_deactivate_children(struct dm_tree_node *dnode,
 				   const char *uuid_prefix,
 				   size_t uuid_prefix_len)
 {
-	return _dm_tree_deactivate_children(dnode, uuid_prefix, uuid_prefix_len, 0);
+	uint32_t lib_cookie;
+	int r;
+
+	if (!_use_lib_cookie(dnode, &lib_cookie))
+		return_0;
+
+	r = _dm_tree_deactivate_children(dnode, uuid_prefix, uuid_prefix_len, 0);
+
+	if (!_wait_on_cookie(dnode, lib_cookie))
+		stack;
+
+	return r;
 }
 
 void dm_tree_skip_lockfs(struct dm_tree_node *dnode)
@@ -1274,9 +1315,9 @@ int dm_tree_suspend_children(struct dm_tree_node *dnode,
 	return r;
 }
 
-int dm_tree_activate_children(struct dm_tree_node *dnode,
-				 const char *uuid_prefix,
-				 size_t uuid_prefix_len)
+static int _dm_tree_activate_children(struct dm_tree_node *dnode,
+				      const char *uuid_prefix,
+				      size_t uuid_prefix_len)
 {
 	int r = 1;
 	void *handle = NULL;
@@ -1297,7 +1338,7 @@ int dm_tree_activate_children(struct dm_tree_node *dnode,
 			continue;
 
 		if (dm_tree_node_num_children(child, 0))
-			if (!dm_tree_activate_children(child, uuid_prefix, uuid_prefix_len))
+			if (!_dm_tree_activate_children(child, uuid_prefix, uuid_prefix_len))
 				return_0;
 	}
 
@@ -1353,7 +1394,23 @@ int dm_tree_activate_children(struct dm_tree_node *dnode,
 		}
 	}
 
-	handle = NULL;
+	return r;
+}
+
+int dm_tree_activate_children(struct dm_tree_node *dnode,
+			      const char *uuid_prefix,
+			      size_t uuid_prefix_len)
+{
+	uint32_t lib_cookie;
+	int r;
+
+	if (!_use_lib_cookie(dnode, &lib_cookie))
+		return_0;
+
+	r = _dm_tree_activate_children(dnode, uuid_prefix, uuid_prefix_len);
+
+	if (!_wait_on_cookie(dnode, lib_cookie))
+		stack;
 
 	return r;
 }
@@ -1839,9 +1896,10 @@ out:
 	return r;
 }
 
-int dm_tree_preload_children(struct dm_tree_node *dnode,
-			     const char *uuid_prefix,
-			     size_t uuid_prefix_len)
+static int _dm_tree_preload_children(struct dm_tree_node *dnode,
+				     const char *uuid_prefix,
+				     size_t uuid_prefix_len,
+				     uint32_t* lib_cookie)
 {
 	int r = 1;
 	void *handle = NULL;
@@ -1861,7 +1919,7 @@ int dm_tree_preload_children(struct dm_tree_node *dnode,
 			continue;
 
 		if (dm_tree_node_num_children(child, 0))
-			if (!dm_tree_preload_children(child, uuid_prefix, uuid_prefix_len))
+			if (!_dm_tree_preload_children(child, uuid_prefix, uuid_prefix_len, lib_cookie))
 				return_0;
 
 		/* FIXME Cope if name exists with no uuid? */
@@ -1913,18 +1971,38 @@ int dm_tree_preload_children(struct dm_tree_node *dnode,
 
 	}
 
-	handle = NULL;
-
 	if (update_devs_flag) {
-		if (!dm_udev_wait(dm_tree_get_cookie(dnode)))
+		if (!_wait_on_cookie(dnode, *lib_cookie))
 			stack;
-		dm_tree_set_cookie(dnode, 0);
+		/* For internal lib_cookie - dm_wait in update_dev() */
 		dm_task_update_nodes();
+
+		/* For internal cookie - request regenerate */
+		if (*lib_cookie && !_use_lib_cookie(dnode, lib_cookie))
+			return_0;
 	}
 
 	return r;
 }
 
+int dm_tree_preload_children(struct dm_tree_node *dnode,
+			     const char *uuid_prefix,
+			     size_t uuid_prefix_len)
+{
+	uint32_t lib_cookie;
+	int r;
+
+	if (!_use_lib_cookie(dnode, &lib_cookie))
+		return_0;
+
+	r = _dm_tree_preload_children(dnode, uuid_prefix, uuid_prefix_len, &lib_cookie);
+
+	if (!_wait_on_cookie(dnode, lib_cookie))
+		stack;
+
+	return r;
+}
+
 /*
  * Returns 1 if unsure.
  */
-- 
1.7.3.3

[PATCH 5/9] Stop calling fs_unlock() and dm_udev_wait()

[Zdenek Kabelac]

Stop calling fs_unlock() from lv_de/activate()

Stop directly calling dm_udev_wait() and
dm_tree_set/get_cookie() from lvm code -
it's now handled internaly by  libdm.
(in fact dm_tree_set/get_cookie is not used any more
and could have been removed - but as it's already public API
I'm not sure if we could remove it ?)

define HAVE_UNLOCK_FS is used for quick compare of how the perfomace
is improved when it's in use.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 lib/activate/activate.c    |    5 +++++
 lib/activate/dev_manager.c |   17 +----------------
 2 files changed, 6 insertions(+), 16 deletions(-)

diff --git a/lib/activate/activate.c b/lib/activate/activate.c
index a19d556..9372033 100644
--- a/lib/activate/activate.c
+++ b/lib/activate/activate.c
@@ -35,6 +35,7 @@
 #include <unistd.h>
 
 #define _skip(fmt, args...) log_very_verbose("Skipping: " fmt , ## args)
+#define HAVE_UNLOCK_FS 1
 
 int lvm1_present(struct cmd_context *cmd)
 {
@@ -1206,7 +1207,9 @@ int lv_deactivate(struct cmd_context *cmd, const char *lvid_s)
 	memlock_inc(cmd);
 	r = _lv_deactivate(lv);
 	memlock_dec(cmd);
+#ifndef HAVE_UNLOCK_FS
 	fs_unlock();
+#endif
 
 	if (!lv_info(cmd, lv, 0, &info, 1, 0) || info.exists)
 		r = 0;
@@ -1305,7 +1308,9 @@ static int _lv_activate(struct cmd_context *cmd, const char *lvid_s,
 	if (!(r = _lv_activate_lv(lv, 0)))
 		stack;
 	memlock_dec(cmd);
+#ifndef HAVE_UNLOCK_FS
 	fs_unlock();
+#endif
 
 	if (r && !monitor_dev_for_events(cmd, lv, 0, 1))
 		stack;
diff --git a/lib/activate/dev_manager.c b/lib/activate/dev_manager.c
index a58b7c5..3b95419 100644
--- a/lib/activate/dev_manager.c
+++ b/lib/activate/dev_manager.c
@@ -1615,7 +1615,6 @@ static int _clean_tree(struct dev_manager *dm, struct dm_tree_node *root, char *
 	struct dm_tree_node *child;
 	char *vgname, *lvname, *layer;
 	const char *name, *uuid;
-	int r;
 
 	while ((child = dm_tree_next_child(&handle, root, 0))) {
 		if (!(name = dm_tree_node_get_name(child)))
@@ -1637,12 +1636,7 @@ static int _clean_tree(struct dev_manager *dm, struct dm_tree_node *root, char *
 		if (non_toplevel_tree_dlid && !strcmp(non_toplevel_tree_dlid, uuid))
 			continue;
 
-		dm_tree_set_cookie(root, 0);
-		r = dm_tree_deactivate_children(root, uuid, strlen(uuid));
-		if (!dm_udev_wait(dm_tree_get_cookie(root)))
-			stack;
-
-		if (!r)
+		if (!dm_tree_deactivate_children(root, uuid, strlen(uuid)))
 			return_0;
 	}
 
@@ -1677,10 +1671,7 @@ static int _tree_action(struct dev_manager *dm, struct logical_volume *lv,
 		break;
 	case DEACTIVATE:
  		/* Deactivate LV and all devices it references that nothing else has open. */
-		dm_tree_set_cookie(root, 0);
 		r = dm_tree_deactivate_children(root, dlid, ID_LEN + sizeof(UUID_PREFIX) - 1);
-		if (!dm_udev_wait(dm_tree_get_cookie(root)))
-			stack;
 		if (!r)
 			goto_out;
 		if (!_remove_lv_symlinks(dm, root))
@@ -1701,10 +1692,7 @@ static int _tree_action(struct dev_manager *dm, struct logical_volume *lv,
 			goto_out;
 
 		/* Preload any devices required before any suspensions */
-		dm_tree_set_cookie(root, 0);
 		r = dm_tree_preload_children(root, dlid, ID_LEN + sizeof(UUID_PREFIX) - 1);
-		if (!dm_udev_wait(dm_tree_get_cookie(root)))
-			stack;
 		if (!r)
 			goto_out;
 
@@ -1712,10 +1700,7 @@ static int _tree_action(struct dev_manager *dm, struct logical_volume *lv,
 			dm->flush_required = 1;
 
 		if (action == ACTIVATE) {
-			dm_tree_set_cookie(root, 0);
 			r = dm_tree_activate_children(root, dlid, ID_LEN + sizeof(UUID_PREFIX) - 1);
-			if (!dm_udev_wait(dm_tree_get_cookie(root)))
-				stack;
 			if (!r)
 				goto_out;
 			if (!_create_lv_symlinks(dm, root)) {
-- 
1.7.3.3

[PATCH 6/9] Add LCK_UNLOCK_FS and unlock_fs

[Zdenek Kabelac]

Use new bit in LCK flags - LCK_UNLOCK_FS
And add new wrapper unlock_fs() around lock_vol for clustering.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 lib/locking/locking.h |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/lib/locking/locking.h b/lib/locking/locking.h
index cb1a55f..df33aec 100644
--- a/lib/locking/locking.h
+++ b/lib/locking/locking.h
@@ -94,6 +94,7 @@ int check_lvm1_vg_inactive(struct cmd_context *cmd, const char *vgname);
 #define LCK_CLUSTER_VG	0x00000080U	/* VG is clustered */
 #define LCK_CACHE	0x00000100U	/* Operation on cache only using P_ lock */
 #define LCK_ORIGIN_ONLY	0x00000200U	/* Operation should bypass any snapshots */
+#define LCK_UNLOCK_FS	0x00000400U	/* Operation waits to settle devices */
 
 /*
  * Additional lock bits for cluster communication via args[1]
@@ -125,6 +126,7 @@ int check_lvm1_vg_inactive(struct cmd_context *cmd, const char *vgname);
 #define LCK_VG_REVERT		(LCK_VG | LCK_READ  | LCK_CACHE | LCK_HOLD)
 
 #define LCK_VG_BACKUP		(LCK_VG | LCK_CACHE)
+#define LCK_VG_UNLOCK_FS       	(LCK_VG | LCK_UNLOCK_FS)
 
 #define LCK_LV_EXCLUSIVE	(LCK_LV | LCK_EXCL)
 #define LCK_LV_SUSPEND		(LCK_LV | LCK_WRITE)
@@ -169,6 +171,8 @@ int check_lvm1_vg_inactive(struct cmd_context *cmd, const char *vgname);
 	lock_vol((vg)->cmd, (vg)->name, LCK_VG_REVERT)
 #define remote_backup_metadata(vg)	\
 	lock_vol((vg)->cmd, (vg)->name, LCK_VG_BACKUP)
+#define unlock_fs(cmd)	\
+	lock_vol(cmd, VG_ORPHANS, LCK_VG_UNLOCK_FS)
 
 /* Process list of LVs */
 int suspend_lvs(struct cmd_context *cmd, struct dm_list *lvs);
-- 
1.7.3.3

[PATCH 7/9] Add CLVMD_CMD_UNLOCK_FS command

[Zdenek Kabelac]

Try to add new UNLOCK_FS command into clvmd.
It's calling lvm_do_unlock_fs() when such command arrives.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 daemons/clvmd/clvm.h          |    1 +
 daemons/clvmd/clvmd-command.c |    6 ++++++
 daemons/clvmd/clvmd.c         |    3 +++
 3 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/daemons/clvmd/clvm.h b/daemons/clvmd/clvm.h
index c9ea10c..b608692 100644
--- a/daemons/clvmd/clvm.h
+++ b/daemons/clvmd/clvm.h
@@ -71,4 +71,5 @@ static const char CLVMD_SOCKNAME[]= DEFAULT_RUN_DIR "/clvmd.sock";
 #define CLVMD_CMD_SET_DEBUG	    42
 #define CLVMD_CMD_VG_BACKUP	    43
 #define CLVMD_CMD_RESTART	    44
+#define CLVMD_CMD_UNLOCK_FS	    45
 #endif
diff --git a/daemons/clvmd/clvmd-command.c b/daemons/clvmd/clvmd-command.c
index 915fcc7..115c149 100644
--- a/daemons/clvmd/clvmd-command.c
+++ b/daemons/clvmd/clvmd-command.c
@@ -147,6 +147,10 @@ int do_command(struct local_client *client, struct clvm_header *msg, int msglen,
 		restart_clvmd();
 		break;
 
+	case CLVMD_CMD_UNLOCK_FS:
+		lvm_do_unlock_fs();
+		break;
+
 	case CLVMD_CMD_GET_CLUSTERNAME:
 		status = clops->get_cluster_name(*buf, buflen);
 		if (!status)
@@ -276,6 +280,7 @@ int do_pre_command(struct local_client *client)
 	case CLVMD_CMD_VG_BACKUP:
 	case CLVMD_CMD_LOCK_QUERY:
 	case CLVMD_CMD_RESTART:
+	case CLVMD_CMD_UNLOCK_FS:
 		break;
 
 	default:
@@ -307,6 +312,7 @@ int do_post_command(struct local_client *client)
 	case CLVMD_CMD_LOCK_VG:
 	case CLVMD_CMD_VG_BACKUP:
 	case CLVMD_CMD_LOCK_QUERY:
+	case CLVMD_CMD_UNLOCK_FS:
 		/* Nothing to do here */
 		break;
 
diff --git a/daemons/clvmd/clvmd.c b/daemons/clvmd/clvmd.c
index 40d7feb..bd9d69c 100644
--- a/daemons/clvmd/clvmd.c
+++ b/daemons/clvmd/clvmd.c
@@ -277,6 +277,9 @@ static const char *decode_cmd(unsigned char cmdl)
 	case CLVMD_CMD_RESTART:
 		command = "RESTART";
 		break;
+	case CLVMD_CMD_UNLOCK_FS:
+		command = "UNLOCK_FS";
+		break;
 	default:
 		command = "unknown";
 		break;
-- 
1.7.3.3

[PATCH 8/9] Locking with unlock_fs

[Zdenek Kabelac]

Add support for LCK_UNLOCK_FS

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 lib/locking/cluster_locking.c |    7 +++++++
 lib/locking/file_locking.c    |    6 ++++++
 2 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/lib/locking/cluster_locking.c b/lib/locking/cluster_locking.c
index eea9974..3ef9cfb 100644
--- a/lib/locking/cluster_locking.c
+++ b/lib/locking/cluster_locking.c
@@ -401,6 +401,13 @@ int lock_resource(struct cmd_context *cmd, const char *resource, uint32_t flags)
 
 	switch (flags & LCK_SCOPE_MASK) {
 	case LCK_VG:
+		/* If unlock fs requested wait until devices are ready */
+		if (flags & LCK_UNLOCK_FS) {
+			log_very_verbose("Requesting unlock fs");
+			return _lock_for_cluster(cmd, CLVMD_CMD_UNLOCK_FS,
+						 LCK_CLUSTER_VG, resource);
+		}
+
 		if (flags == LCK_VG_BACKUP) {
 			log_very_verbose("Requesting backup of VG metadata for %s",
 					 resource);
diff --git a/lib/locking/file_locking.c b/lib/locking/file_locking.c
index ed1ccd5..8461140 100644
--- a/lib/locking/file_locking.c
+++ b/lib/locking/file_locking.c
@@ -258,6 +258,12 @@ static int _file_lock_resource(struct cmd_context *cmd, const char *resource,
 
 	switch (flags & LCK_SCOPE_MASK) {
 	case LCK_VG:
+		if (flags & LCK_UNLOCK_FS) {
+			/* Wait until devices are ready */
+			fs_unlock();
+			break;
+		}
+
 		/* Skip cache refresh for VG_GLOBAL - the caller handles it */
 		if (strcmp(resource, VG_GLOBAL))
 			lvmcache_drop_metadata(resource, 0);
-- 
1.7.3.3

[PATCH 9/9] Wait for devices being created

[Zdenek Kabelac]

Before command's exit and before device is needed for open in set_lv().

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 lib/metadata/lv_manip.c |    3 +++
 tools/lvmcmdline.c      |    3 +++
 2 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/lib/metadata/lv_manip.c b/lib/metadata/lv_manip.c
index 6f553cd..48e4b7c 100644
--- a/lib/metadata/lv_manip.c
+++ b/lib/metadata/lv_manip.c
@@ -2999,6 +2999,9 @@ int set_lv(struct cmd_context *cmd, struct logical_volume *lv,
 	struct device *dev;
 	char *name;
 
+	if (!unlock_fs(cmd)) /* waits until device is ready for regular open */
+		stack;
+
 	/*
 	 * FIXME:
 	 * <clausen> also, more than 4k
diff --git a/tools/lvmcmdline.c b/tools/lvmcmdline.c
index 9f20d4b..2fba753 100644
--- a/tools/lvmcmdline.c
+++ b/tools/lvmcmdline.c
@@ -1117,6 +1117,9 @@ int lvm_run_command(struct cmd_context *cmd, int argc, char **argv)
 
 	ret = cmd->command->fn(cmd, argc, argv);
 
+	if (!unlock_fs(cmd))
+		stack;
+
 	fin_locking();
 
       out:
-- 
1.7.3.3

[PATCH 0/9] Udev latency

[Alasdair G Kergon]

On Wed, Dec 08, 2010 at 01:57:46PM +0100, Zdenek Kabelac wrote:
> WARNING  - Not a final patch - this is patch for discussion WARNING
 
We had a discussion about this yesterday.

Zdenek is working on an updated patchset which addresses all the issues
we could come up with.

Summary of the (revised) patchset:
- Reuse a single cookie for consecutive operations instead of creating
a new one every time.
- Release the cookie (with fs_unlock etc.) only when unlocking the VG (avoids
need for extra cluster message and avoids races between commands if it were
done later).  
- Only do the udev_wait when an 'immediate' dev is encountered during tree
processing (cmirror) and when an LV just activated needs to be opened (just
set_lv needs this).

Alasdair