[PATCH 0/4] Valgrind fixes

[PATCH 0/4] Valgrind fixes

[Zdenek Kabelac]

Another small set mainly with fixes discover with valgrind.
I assume patch2 is the really important one.

Zdenek Kabelac (4):
  Reading of unitialized memory
  Fix access to released memory
  Another fix for garbage send in clvmd message
  Better shutdown for clvmd

 daemons/clvmd/clvmd.c         |   30 ++++++++++++++++++++++--------
 daemons/clvmd/refresh_clvmd.c |    5 +++--
 lib/cache/lvmcache.c          |   24 ++++++++++++++----------
 lib/format1/import-extents.c  |    2 +-
 4 files changed, 40 insertions(+), 21 deletions(-)

-- 
1.7.4.1

[PATCH 1/4] Reading of unitialized memory

[Zdenek Kabelac]

This is rather hint fix just for reading of unknown memory.

Could be reached via few our test cases:

==11501== Invalid read of size 8
==11501==    at 0x49B2E0: _area_length (import-extents.c:204)
==11501==    by 0x49B40C: _read_linear (import-extents.c:222)
==11501==    by 0x49B952: _build_segments (import-extents.c:323)
==11501==    by 0x49B9A0: _build_all_segments (import-extents.c:334)
==11501==    by 0x49BB4C: import_extents (import-extents.c:364)
==11501==    by 0x497655: _format1_vg_read (format1.c:217)
==11501==    by 0x47E43E: _vg_read (metadata.c:2901)

cut from t-vgcvgbackup-usage.sh
--
pvcreate -M1 $(cat DEVICES)
vgcreate -M1 -c n $vg $(cat DEVICES)
lvcreate -l1 -n $lv1 $vg $dev1
--

Idea of the fix is rather defensive - to allocate one extra element
to 'map' array which is then used in _area_length().

lv->le_count == 1 - thus allocated 'map' array allows to access only
element [0] - but the loop in _area_lenght start directly with:
 len++ (== 1) - thus accessing element behind the allocated array.

As I'm not familiar with the LVM1 code so there is potentialy better fix.
My patch is rather hint where is the bug - and avoids read of random
memory - but does not fix algorithmic bug here - do we even care about
format 1 ?

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 lib/format1/import-extents.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/format1/import-extents.c b/lib/format1/import-extents.c
index 99723ee..dc12776 100644
--- a/lib/format1/import-extents.c
+++ b/lib/format1/import-extents.c
@@ -64,7 +64,7 @@ static struct dm_hash_table *_create_lv_maps(struct dm_pool *mem,
 
 		lvm->lv = ll->lv;
 		if (!(lvm->map = dm_pool_zalloc(mem, sizeof(*lvm->map)
-					     * ll->lv->le_count)))
+					     * (ll->lv->le_count + 1))))
 			goto_bad;
 
 		if (!dm_hash_insert(maps, ll->lv->name, lvm))
-- 
1.7.4.1

[PATCH 2/4] Fix access to released memory

[Zdenek Kabelac]

Somewhat tricky part of the code.

Invalid primary_vginfo  was supposed to move all its lvmcache_info to
orphan_vginfo - however it has called _drop_vginfo() inside the loop
that remove primary_vginfo itself - thus made the loop using released
memory.

Patch used _vginfo_detach_info() instead and call _drop_vginfo after
this whole loop is finished.

Here is valgrind trace it should be fixed:

Invalid read of size 8
   at 0x41E960: _lvmcache_update_vgname (lvmcache.c:1229)
   by 0x41EF86: lvmcache_update_vgname_and_id (lvmcache.c:1360)
   by 0x441393: _text_read (text_label.c:329)
   by 0x442221: label_read (label.c:289)
   by 0x41CF92: lvmcache_label_scan (lvmcache.c:635)
   by 0x45B303: _vg_read_by_vgid (metadata.c:3342)
   by 0x45B4A6: lv_from_lvid (metadata.c:3381)
   by 0x41B555: lv_activation_filter (activate.c:1346)
   by 0x415868: do_activate_lv (lvm-functions.c:343)
   by 0x415E8C: do_lock_lv (lvm-functions.c:532)
   by 0x40FD5F: do_command (clvmd-command.c:120)
   by 0x413D7B: process_local_command (clvmd.c:1686)
 Address 0x63eba10 is 16 bytes inside a block of size 160 free'd
   at 0x4C2756E: free (vg_replace_malloc.c:366)
   by 0x41DE70: _free_vginfo (lvmcache.c:980)
   by 0x41DEDA: _drop_vginfo (lvmcache.c:998)
   by 0x41E854: _lvmcache_update_vgname (lvmcache.c:1238)
   by 0x41EF86: lvmcache_update_vgname_and_id (lvmcache.c:1360)
   by 0x441393: _text_read (text_label.c:329)
   by 0x442221: label_read (label.c:289)
   by 0x41CF92: lvmcache_label_scan (lvmcache.c:635)
   by 0x45B303: _vg_read_by_vgid (metadata.c:3342)
   by 0x45B4A6: lv_from_lvid (metadata.c:3381)
   by 0x41B555: lv_activation_filter (activate.c:1346)
   by 0x415868: do_activate_lv (lvm-functions.c:343)

problematic line:
dm_list_iterate_items_safe(info2, info3, &primary_vginfo->infos)

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 lib/cache/lvmcache.c |   24 ++++++++++++++----------
 1 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/lib/cache/lvmcache.c b/lib/cache/lvmcache.c
index 4e2e158..ee7d278 100644
--- a/lib/cache/lvmcache.c
+++ b/lib/cache/lvmcache.c
@@ -1107,17 +1107,17 @@ static int _lvmcache_update_vgname(struct lvmcache_info *info,
 		 * Otherwise we risk bogus warnings of duplicate VGs.
 		 */
 		while ((primary_vginfo = vginfo_from_vgname(vgname, NULL)) &&
-		       _scanning_in_progress && _vginfo_is_invalid(primary_vginfo))
+		       _scanning_in_progress && _vginfo_is_invalid(primary_vginfo)) {
+			orphan_vginfo = vginfo_from_vgname(primary_vginfo->fmt->orphan_vg_name, NULL);
+			if (!orphan_vginfo) {
+				log_error(INTERNAL_ERROR "Orphan vginfo %s lost from cache.",
+					  primary_vginfo->fmt->orphan_vg_name);
+				dm_free(vginfo->vgname);
+				dm_free(vginfo);
+				return 0;
+			}
 			dm_list_iterate_items_safe(info2, info3, &primary_vginfo->infos) {
-				orphan_vginfo = vginfo_from_vgname(primary_vginfo->fmt->orphan_vg_name, NULL);
-				if (!orphan_vginfo) {
-					log_error(INTERNAL_ERROR "Orphan vginfo %s lost from cache.",
-						  primary_vginfo->fmt->orphan_vg_name);
-					dm_free(vginfo->vgname);
-					dm_free(vginfo);
-					return 0;
-				}
-				_drop_vginfo(info2, primary_vginfo);	
+				_vginfo_detach_info(info2);
 				_vginfo_attach_info(orphan_vginfo, info2);
 				if (info2->mdas.n)
 					sprintf(mdabuf, " with %u mdas",
@@ -1129,6 +1129,10 @@ static int _lvmcache_update_vgname(struct lvmcache_info *info,
 					  vgname, orphan_vginfo->vgid[0] ? " (" : "",
 					  orphan_vginfo->vgid[0] ? orphan_vginfo->vgid : "",
 					  orphan_vginfo->vgid[0] ? ")" : "", mdabuf);
+			}
+
+			if (!_drop_vginfo(NULL, primary_vginfo))
+				return_0;
 		}
 
 		if (!_insert_vginfo(vginfo, vgid, vgstatus, creation_host,
-- 
1.7.4.1

[PATCH 3/4] Another fix for garbage send in clvmd message

[Zdenek Kabelac]

(Similar to my 2 other previous already posted patches).

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 daemons/clvmd/refresh_clvmd.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/daemons/clvmd/refresh_clvmd.c b/daemons/clvmd/refresh_clvmd.c
index 1e88b5c..805fbe3 100644
--- a/daemons/clvmd/refresh_clvmd.c
+++ b/daemons/clvmd/refresh_clvmd.c
@@ -154,6 +154,7 @@ static void _build_header(struct clvm_header *head, int cmd, const char *node,
 	head->cmd = cmd;
 	head->status = 0;
 	head->flags = 0;
+	head->xid = 0;
 	head->clientid = 0;
 	head->arglen = len;
 
@@ -197,11 +198,11 @@ static int _cluster_request(char cmd, const char *node, void *data, int len,
 	if (_clvmd_sock == -1)
 		return 0;
 
-	_build_header(head, cmd, node, len);
+	_build_header(head, cmd, node, len - 1);
 	memcpy(head->node + strlen(head->node) + 1, data, len);
 
 	status = _send_request(outbuf, sizeof(struct clvm_header) +
-			       strlen(head->node) + len, &retbuf, no_response);
+			       strlen(head->node) + len - 1, &retbuf, no_response);
 	if (!status || no_response)
 		goto out;
 
-- 
1.7.4.1

[PATCH 4/4] Better shutdown for clvmd

[Zdenek Kabelac]

Ok - this is 'a small step' towards cleaner shutdown sequence.
It could be seens as unneeded - as normally clvmd doens't care about
unreleased memory on exit - but for valgrind testing it's better to
have them cleaned all.

So - few things are left on exit path - this patch starts to remove
just some of them.

1. lvm_thread_fs is made as a thread which could be joined on exit()
2. memory allocated to local_clien_head list is released.
   (this part is somewhat more complex if the proper reaction is
   needed - and as it requires some heavier code moving - it will
   be resolved later.

Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
---
 daemons/clvmd/clvmd.c |   30 ++++++++++++++++++++++--------
 1 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/daemons/clvmd/clvmd.c b/daemons/clvmd/clvmd.c
index 361fc62..fc092cf 100644
--- a/daemons/clvmd/clvmd.c
+++ b/daemons/clvmd/clvmd.c
@@ -103,8 +103,6 @@ static int child_pipe[2];
 
 typedef enum {IF_AUTO, IF_CMAN, IF_GULM, IF_OPENAIS, IF_COROSYNC, IF_SINGLENODE} if_type_t;
 
-typedef void *(lvm_pthread_fn_t)(void*);
-
 /* Prototypes for code further down */
 static void sigusr2_handler(int sig);
 static void sighup_handler(int sig);
@@ -134,7 +132,7 @@ static int check_all_clvmds_running(struct local_client *client);
 static int local_rendezvous_callback(struct local_client *thisfd, char *buf,
 				     int len, const char *csid,
 				     struct local_client **new_client);
-static void lvm_thread_fn(void *) __attribute__ ((noreturn));
+static void *lvm_thread_fn(void *);
 static int add_to_lvmqueue(struct local_client *client, struct clvm_header *msg,
 			   int msglen, const char *csid);
 static int distribute_command(struct local_client *thisfd);
@@ -333,7 +331,7 @@ static void check_permissions(void)
 int main(int argc, char *argv[])
 {
 	int local_sock;
-	struct local_client *newfd;
+	struct local_client *newfd, *delfd;
 	struct utsname nodeinfo;
 	struct lvm_startup_params lvm_params;
 	int opt;
@@ -581,8 +579,7 @@ int main(int argc, char *argv[])
 	pthread_mutex_lock(&lvm_start_mutex);
 	lvm_params.using_gulm = using_gulm;
 	lvm_params.argv = argv;
-	pthread_create(&lvm_thread, NULL, (lvm_pthread_fn_t*)lvm_thread_fn,
-			(void *)&lvm_params);
+	pthread_create(&lvm_thread, NULL, lvm_thread_fn, &lvm_params);
 
 	/* Tell the rest of the cluster our version number */
 	/* CMAN can do this immediately, gulm needs to wait until
@@ -601,9 +598,24 @@ int main(int argc, char *argv[])
 	/* Do some work */
 	main_loop(local_sock, cmd_timeout);
 
+	pthread_mutex_lock(&lvm_thread_mutex);
+	pthread_cond_signal(&lvm_thread_cond);
+	pthread_mutex_unlock(&lvm_thread_mutex);
+	if ((errno = pthread_join(lvm_thread, NULL)))
+		log_sys_error("pthread_join", "");
+
 	close_local_sock(local_sock);
 	destroy_lvm();
 
+	for (newfd = local_client_head.next; newfd != NULL;) {
+		delfd = newfd;
+		newfd = newfd->next;
+		/* FIXME: needs cleanup code from read_from_local_sock() */
+                /* for now break of CLVMD presents access to free memory here */
+		safe_close(&(delfd->fd));
+		free(delfd);
+	}
+
 	return 0;
 }
 
@@ -1931,7 +1943,7 @@ static int process_work_item(struct lvm_thread_cmd *cmd)
 /*
  * Routine that runs in the "LVM thread".
  */
-static void lvm_thread_fn(void *arg)
+static void *lvm_thread_fn(void *arg)
 {
 	struct dm_list *cmdl, *tmp;
 	sigset_t ss;
@@ -1952,7 +1964,7 @@ static void lvm_thread_fn(void *arg)
 	pthread_mutex_unlock(&lvm_start_mutex);
 
 	/* Now wait for some actual work */
-	for (;;) {
+	while (!quit) {
 		DEBUGLOG("LVM thread waiting for work\n");
 
 		pthread_mutex_lock(&lvm_thread_mutex);
@@ -1975,6 +1987,8 @@ static void lvm_thread_fn(void *arg)
 		}
 		pthread_mutex_unlock(&lvm_thread_mutex);
 	}
+
+	pthread_exit(NULL);
 }
 
 /* Pass down some work to the LVM thread */
-- 
1.7.4.1

[PATCH 3/4] Another fix for garbage send in clvmd message

[Milan Broz]

On 03/24/2011 12:16 PM, Zdenek Kabelac wrote:

> -	_build_header(head, cmd, node, len);
> +	_build_header(head, cmd, node, len - 1);
>  	memcpy(head->node + strlen(head->node) + 1, data, len);
>  
>  	status = _send_request(outbuf, sizeof(struct clvm_header) +
> -			       strlen(head->node) + len, &retbuf, no_response);
> +			       strlen(head->node) + len - 1, &retbuf, no_response);

ACK, but this protocol _MUST_ be rewritten soon completely.

really, no comment to use of this struct...
(note that if strlen(node) > 0, args points somewhere inside on node name. wonderful.

struct clvm_header {
	uint8_t  cmd;	        /* See below */
...
	char node[1];		/* Actually a NUL-terminated string, node name.
				   If this is empty then the command is 
				   forwarded to all cluster nodes unless 
				   FLAG_LOCAL is also set. */
	char args[1];		/* Arguments for the command follow the 
				   node name, This member is only
				   valid if the node name is empty */

Milan

[PATCH 1/4] Reading of unitialized memory

[Milan Broz]

On 03/24/2011 12:16 PM, Zdenek Kabelac wrote:

>  		if (!(lvm->map = dm_pool_zalloc(mem, sizeof(*lvm->map)
> -					     * ll->lv->le_count)))
> +					     * (ll->lv->le_count + 1))))

I do not like such fixes, but because this code is really not going
to be used in future (and I know how complicated it is), let's
"fix" it this way.

turn a blind eye to ... ACK :-)

Milan

[PATCH 2/4] Fix access to released memory

[Milan Broz]

On 03/24/2011 12:16 PM, Zdenek Kabelac wrote:

> Invalid primary_vginfo  was supposed to move all its lvmcache_info to
> orphan_vginfo - however it has called _drop_vginfo() inside the loop
> that remove primary_vginfo itself - thus made the loop using released
> memory.

ack (we talked about it yesterday)

Milan

[PATCH 4/4] Better shutdown for clvmd

[Milan Broz]

On 03/24/2011 12:16 PM, Zdenek Kabelac wrote:
> Ok - this is 'a small step' towards cleaner shutdown sequence.

(Yes it is small step. I would like just mention that we are not
walking on the Moon, but trying to move an elephant from porcelain
storage room.)

Btw there are 3 problems in this patch
- unrelated code tidy
- thread join
- fix memleak and close of descriptors


> -typedef void *(lvm_pthread_fn_t)(void*);
> +static void *lvm_thread_fn(void *);

> -	pthread_create(&lvm_thread, NULL, (lvm_pthread_fn_t*)lvm_thread_fn,
> -			(void *)&lvm_params);
> +	pthread_create(&lvm_thread, NULL, lvm_thread_fn, &lvm_params);

code tidying, please use separate patch for these next time,
so it is not mixed with real bugfixes... ;-)

> +	pthread_mutex_lock(&lvm_thread_mutex);
> +	pthread_cond_signal(&lvm_thread_cond);
> +	pthread_mutex_unlock(&lvm_thread_mutex);
> +	if ((errno = pthread_join(lvm_thread, NULL)))
> +		log_sys_error("pthread_join", "");

> @@ -1952,7 +1964,7 @@ static void lvm_thread_fn(void *arg)
>  	pthread_mutex_unlock(&lvm_start_mutex);
>  
>  	/* Now wait for some actual work */
> -	for (;;) {
> +	while (!quit) {

I hope I thought what can happen when there is still work in queue
and I think it is correctly handled...

ACK

> +	for (newfd = local_client_head.next; newfd != NULL;) {
> +		delfd = newfd;
> +		newfd = newfd->next;
> +		/* FIXME: needs cleanup code from read_from_local_sock() */
> +                /* for now break of CLVMD presents access to free memory here */
> +		safe_close(&(delfd->fd));
> +		free(delfd);
> +	}

Well, this looks safe. Everything should be stopped, so even there is FIXME still,
it fixes part of problem.

ACK

Milan