Linux LVM users
 help / color / mirror / Atom feed
* [linux-lvm] How to monitor and possibly autoextend snapshots
@ 2008-10-24  9:33 marcin.kaluza
  2008-10-24 16:30 ` Jonathan Brassow
  2008-10-28  6:43 ` [linux-lvm] libdlm.h: No such file or directory Stepan Kadlec
  0 siblings, 2 replies; 10+ messages in thread
From: marcin.kaluza @ 2008-10-24  9:33 UTC (permalink / raw)
  To: linux-lvm

Hi

Is there a way to get notified, when the snapshot fills up to a certain
level? i.e.  via dmsetup wait [] ? I looked for event numbers but couldn't
find any doc for them.
The best thing would be to make snapshots autoextensible when given a max
size and resize value (just like oracle tablespaces do), but I suppose
that would require much coding...
Any thoughts?

We're using lvm snapshots for samba shadow copies and database cloning,
but there's a problem with monitoring snapshots % of usage - now we just
have a daemon that polls lvdisplay periodically and extends the lv when
necessary. But this is unreliable - you have to either poll often or keep
enough of free space and still you can end up with a trashed snap.

Martin

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [linux-lvm] How to monitor and possibly autoextend snapshots
  2008-10-24  9:33 [linux-lvm] How to monitor and possibly autoextend snapshots marcin.kaluza
@ 2008-10-24 16:30 ` Jonathan Brassow
  2008-10-27 12:17   ` Marcin Kałuża
  2008-10-28  6:43 ` [linux-lvm] libdlm.h: No such file or directory Stepan Kadlec
  1 sibling, 1 reply; 10+ messages in thread
From: Jonathan Brassow @ 2008-10-24 16:30 UTC (permalink / raw)
  To: LVM general discussion and development

Right now, you get notified when the snapshot becomes invalid.  I  
thought I heard someone talking about adding forewarning, but I don't  
know where that stands.

You can respond to events (like the snapshot going invalid) through  
dmeventd, but any solution like that would require new code... and you  
would also need new code in the kernel to do the forewarning.

Since you have some knowledge of the arguments given to other programs  
for this feature... :)  Perhaps if you would be willing to envision  
the arguments necessary to 'lvcreate' when creating the snapshot, we  
could get the ball rolling.

  brassow


On Oct 24, 2008, at 4:33 AM, marcin.kaluza@comarch.pl wrote:

> Hi
>
> Is there a way to get notified, when the snapshot fills up to a  
> certain
> level? i.e.  via dmsetup wait [] ? I looked for event numbers but  
> couldn't
> find any doc for them.
> The best thing would be to make snapshots autoextensible when given  
> a max
> size and resize value (just like oracle tablespaces do), but I suppose
> that would require much coding...
> Any thoughts?
>
> We're using lvm snapshots for samba shadow copies and database  
> cloning,
> but there's a problem with monitoring snapshots % of usage - now we  
> just
> have a daemon that polls lvdisplay periodically and extends the lv  
> when
> necessary. But this is unreliable - you have to either poll often or  
> keep
> enough of free space and still you can end up with a trashed snap.
>
> Martin
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: [linux-lvm] How to monitor and possibly autoextend snapshots
  2008-10-24 16:30 ` Jonathan Brassow
@ 2008-10-27 12:17   ` Marcin Kałuża
  2008-10-28 20:50     ` Jonathan Brassow
  0 siblings, 1 reply; 10+ messages in thread
From: Marcin Kałuża @ 2008-10-27 12:17 UTC (permalink / raw)
  To: 'LVM general discussion and development'

Hi,
Well, that's not much of help since after it becomes invalid I'll know it
anyway (IO error ;D), so the event is purely informational.
The "other program" is just my simple script, that periodically scans all
mounted snapshots, checks if their root has specific xattrs set ("increase"
and "maxsize" - drawback is that it has to be mounted to work, the bonus is
that it is mountpoint and configuration file independent) and based on those
parameters it polls and resizes those snapshots if necessary. The problem
with this script is that if it polls them not often enough, there is a
possibility that it won't resize it in time.

The simplest (and mostly sufficient) thing to do would be forewarning that
you talk about. 

The way I see it:
In it's simplest form, at snapshot creation (or later of course) you would
give an extra param like --warn-level with either absolute size or
percentage value, so that lvm emits an event when free space in the snapshot
goes below that value. 
I think this doesn't require much work.

Better solution would be to have another two parameters, like --increase-by
and --max-size, that would tell lvm to enlarge the snapshot by --increase-by
MB's or percent when free space goes below --warn-limit(it could emit the
event anyway). --max-size would be enlargment limit, because sometimes we
might prefer snapshot to become invalid than to eat up too much space in vg.
Of course without --max-size the snapshot would extend until reaching the
size of original lv.
But I suppose this requires significantly more coding.

greetz,
Martin

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] On
Behalf Of Jonathan Brassow
Sent: Friday, October 24, 2008 6:30 PM
To: LVM general discussion and development
Subject: Re: [linux-lvm] How to monitor and possibly autoextend snapshots

Right now, you get notified when the snapshot becomes invalid.  I  
thought I heard someone talking about adding forewarning, but I don't  
know where that stands.

You can respond to events (like the snapshot going invalid) through  
dmeventd, but any solution like that would require new code... and you  
would also need new code in the kernel to do the forewarning.

Since you have some knowledge of the arguments given to other programs  
for this feature... :)  Perhaps if you would be willing to envision  
the arguments necessary to 'lvcreate' when creating the snapshot, we  
could get the ball rolling.

  brassow


On Oct 24, 2008, at 4:33 AM, marcin.kaluza@comarch.pl wrote:

> Hi
>
> Is there a way to get notified, when the snapshot fills up to a  
> certain
> level? i.e.  via dmsetup wait [] ? I looked for event numbers but  
> couldn't
> find any doc for them.
> The best thing would be to make snapshots autoextensible when given  
> a max
> size and resize value (just like oracle tablespaces do), but I suppose
> that would require much coding...
> Any thoughts?
>
> We're using lvm snapshots for samba shadow copies and database  
> cloning,
> but there's a problem with monitoring snapshots % of usage - now we  
> just
> have a daemon that polls lvdisplay periodically and extends the lv  
> when
> necessary. But this is unreliable - you have to either poll often or  
> keep
> enough of free space and still you can end up with a trashed snap.
>
> Martin
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [linux-lvm] libdlm.h: No such file or directory
  2008-10-24  9:33 [linux-lvm] How to monitor and possibly autoextend snapshots marcin.kaluza
  2008-10-24 16:30 ` Jonathan Brassow
@ 2008-10-28  6:43 ` Stepan Kadlec
  2008-10-28  8:30   ` Christine Caulfield
  1 sibling, 1 reply; 10+ messages in thread
From: Stepan Kadlec @ 2008-10-28  6:43 UTC (permalink / raw)
  To: linux-lvm

hello list,
I am trying to compile clustered LVM with OpenAIS (instead of cluster 
framework) using

./configure --with-clvmd=openais --with-cluster=shared --libdir=/usr/lib64/

but it fails because of missing libdlm.h. I know this is part of the 
cluster framework, but I thought I don't need 'cluster' when using 
'OpenAIS'.

what is the clean way for compiling clustered LVM with OpenAIS?

thanks for any hints. stepan

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [linux-lvm] libdlm.h: No such file or directory
  2008-10-28  6:43 ` [linux-lvm] libdlm.h: No such file or directory Stepan Kadlec
@ 2008-10-28  8:30   ` Christine Caulfield
  0 siblings, 0 replies; 10+ messages in thread
From: Christine Caulfield @ 2008-10-28  8:30 UTC (permalink / raw)
  To: LVM general discussion and development

Stepan Kadlec wrote:
> hello list,
> I am trying to compile clustered LVM with OpenAIS (instead of cluster
> framework) using
> 
> ./configure --with-clvmd=openais --with-cluster=shared --libdir=/usr/lib64/
> 
> but it fails because of missing libdlm.h. I know this is part of the
> cluster framework, but I thought I don't need 'cluster' when using
> 'OpenAIS'.
> 
> what is the clean way for compiling clustered LVM with OpenAIS?
> 

openais support is not included in the configure script for clvmd yet.
This is mainly because the locking support in openais still needs some
work to make it totally reliable.

If you want to test clvmd over just openais then you will need to
hand-edit the Makefile for the moment ... or sweet-talk someone into
fixing the configure script ;-)

Chrissie

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [linux-lvm] How to monitor and possibly autoextend snapshots
  2008-10-27 12:17   ` Marcin Kałuża
@ 2008-10-28 20:50     ` Jonathan Brassow
  2008-10-29 11:10       ` Marcin Kałuża
  2008-10-29 14:26       ` Marcin Kałuża
  0 siblings, 2 replies; 10+ messages in thread
From: Jonathan Brassow @ 2008-10-28 20:50 UTC (permalink / raw)
  To: LVM general discussion and development

[-- Attachment #1: Type: text/plain, Size: 988 bytes --]

On Oct 27, 2008, at 7:17 AM, Marcin Kałuża wrote:

> Hi,
> Well, that's not much of help since after it becomes invalid I'll  
> know it
> anyway (IO error ;D), so the event is purely informational.


Well, I've thrown together a quick patch that should give you what you  
want.  (I have not compiled/tested it.)  We'd have to give it more  
thought for it to become a permanent feature (or get integrated into  
LVM), but it should be a nice little hack.  If you turn out to like  
the idea, maybe more discussion will evolve.

You'll have to compile your own kernel with the patch applied, but  
once you've done so, you should be able to do the following:

# Get snapshot to raise an event when 75% full
prompt> dmsetup message <snapshot device> 0 notify 75
prompt> dmsetup wait <snapshot device>
# Ok, now we are 75% full - let me know when 95% full
prompt> dmsetup message <snapshot device> 0 notify 95
prompt> dmsetup wait <snapshot device>

  brassow


[-- Attachment #2: snap.patch --]
[-- Type: application/octet-stream, Size: 2320 bytes --]

diff --git a/drivers/md/dm-snap.c b/drivers/md/dm-snap.c
index b2d9d1a..f827c2a 100644
--- a/drivers/md/dm-snap.c
+++ b/drivers/md/dm-snap.c
@@ -890,6 +890,17 @@ static void pending_complete(struct dm_snap_pending_exception *pe, int success)
 
 	up_write(&s->lock);
 
+	if ((s->event_percent) && s->store.fraction_full) {
+		sector_t numerator, denominator;
+		snap->store.fraction_full(&snap->store, &numerator,
+					  &denominator);
+		numerator *= 100;
+		if (s->event_percent < dm_sector_div_up(numerator, denominator)) {
+			s->event_percent = 0;
+			dm_table_event(s->ti->table);
+		}
+	}
+
 	/* Submit any pending write bios */
 	if (error)
 		error_bios(snapshot_bios);
@@ -1151,6 +1162,31 @@ static int snapshot_status(struct dm_target *ti, status_type_t type,
 	return 0;
 }
 
+static int snapshot_message(struct dm_target *ti, unsigned argc, char **argv)
+{
+	struct dm_snapshot *snap = ti->private;
+
+	if (argc != 2) {
+		DMERR("Message to snapshot received, but two args are required");
+		return 0;
+	}
+
+	if (strncmp(argv[0], "notify")) {
+		DMERR("Message to snapshot received, but should be 'notify <%%>'");
+		return 0;
+	}
+
+	if (sscanf(argv[1], "%d", &snap->event_percent) != 1) {
+		DMERR("Message to snapshot received, but should be 'notify <%%>'");
+		return 0;
+	}
+
+	if ((snap->event_percent < 0) || (snap->event_percent > 99)) {
+		DMERR("Invalid notification percentage to snapshot - resetting to 0");
+		snap->event_percent = 0;
+	}
+}
+
 /*-----------------------------------------------------------------
  * Origin methods
  *---------------------------------------------------------------*/
@@ -1382,6 +1418,7 @@ static struct target_type snapshot_target = {
 	.end_io  = snapshot_end_io,
 	.resume  = snapshot_resume,
 	.status  = snapshot_status,
+	.message = snapshot_message,
 };
 
 static int __init dm_snapshot_init(void)
diff --git a/drivers/md/dm-snap.h b/drivers/md/dm-snap.h
index f07315f..15ed4c1 100644
--- a/drivers/md/dm-snap.h
+++ b/drivers/md/dm-snap.h
@@ -149,7 +149,8 @@ struct dm_snapshot {
 	chunk_t chunk_mask;
 	chunk_t chunk_shift;
 
-	/* You can't use a snapshot if this is 0 (e.g. if full) */
+	/* You can't use a snapshot if 'valid' is 0 (e.g. if full) */
+	int event_percent;
 	int valid;
 
 	/* Origin writes don't trigger exceptions until this is set */

[-- Attachment #3: Type: text/plain, Size: 3 bytes --]





^ permalink raw reply related	[flat|nested] 10+ messages in thread

* RE: [linux-lvm] How to monitor and possibly autoextend snapshots
  2008-10-28 20:50     ` Jonathan Brassow
@ 2008-10-29 11:10       ` Marcin Kałuża
  2008-10-29 14:26       ` Marcin Kałuża
  1 sibling, 0 replies; 10+ messages in thread
From: Marcin Kałuża @ 2008-10-29 11:10 UTC (permalink / raw)
  To: 'LVM general discussion and development'

[-- Attachment #1: Type: text/plain, Size: 2275 bytes --]

Wow - this was quick ;))

I tried to compile it with kernel-2.6.18-92.1.13.el5.centos.plus kernel and
it threw 3 errors:

1)
drivers/md/dm-snap.c:792: error: snap undeclared (first use in this
function)
2)
drivers/md/dm-snap.c:797: error: struct dm_snapshot has no member named ti
3)
drivers/md/dm-snap.c: In function snapshot_message:
drivers/md/dm-snap.c:1066: error: too few arguments to function strncmp

1)
There was:
snap->store.fraction_full(&snap->store, &numerator,
and I think it should be
s->store.fraction_full(&s->store, &numerator,

3)
There was
if (strncmp(argv[0], "notify")) {
changed to:
if (strncmp(argv[0], "notify", 6)) {

2)
dm_table_event(s->ti->table);
changed to 
dm_table_event(s->table);
or should there be "ti" field in the dm_snapshot ?

But now I'm not sure if it'll do what it's meant to do ;)
It compiles. As soon as I set up some test vm with this kernel I'll let you
know if it works.

Changed patch is enclosed.

Martin
-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] On
Behalf Of Jonathan Brassow
Sent: Tuesday, October 28, 2008 9:51 PM
To: LVM general discussion and development
Subject: Re: [linux-lvm] How to monitor and possibly autoextend snapshots

On Oct 27, 2008, at 7:17 AM, Marcin Kałuża wrote:

> Hi,
> Well, that's not much of help since after it becomes invalid I'll  
> know it
> anyway (IO error ;D), so the event is purely informational.


Well, I've thrown together a quick patch that should give you what you  
want.  (I have not compiled/tested it.)  We'd have to give it more  
thought for it to become a permanent feature (or get integrated into  
LVM), but it should be a nice little hack.  If you turn out to like  
the idea, maybe more discussion will evolve.

You'll have to compile your own kernel with the patch applied, but  
once you've done so, you should be able to do the following:

# Get snapshot to raise an event when 75% full
prompt> dmsetup message <snapshot device> 0 notify 75
prompt> dmsetup wait <snapshot device>
# Ok, now we are 75% full - let me know when 95% full
prompt> dmsetup message <snapshot device> 0 notify 95
prompt> dmsetup wait <snapshot device>

  brassow


[-- Attachment #2: snap.patch --]
[-- Type: application/octet-stream, Size: 2389 bytes --]

diff --git a/drivers/md/dm-snap.c b/drivers/md/dm-snap.c
index b2d9d1a..f827c2a 100644
--- a/drivers/md/dm-snap.c
+++ b/drivers/md/dm-snap.c
@@ -890,6 +890,17 @@ static void pending_complete(struct dm_snap_pending_exception *pe, int success)
 
 	up_write(&s->lock);
 
+	if ((s->event_percent) && s->store.fraction_full) {
+		sector_t numerator, denominator;
+		s->store.fraction_full(&s->store, &numerator,
+					  &denominator);
+		numerator *= 100;
+		if (s->event_percent < dm_sector_div_up(numerator, denominator)) {
+			s->event_percent = 0;
+			dm_table_event(s->table);
+		}
+	}
+
 	/* Submit any pending write bios */
 	if (error)
 		error_bios(snapshot_bios);
@@ -1151,6 +1162,31 @@ static int snapshot_status(struct dm_target *ti, status_type_t type,
 	return 0;
 }
 
+static int snapshot_message(struct dm_target *ti, unsigned argc, char **argv)
+{
+	struct dm_snapshot *snap = ti->private;
+
+	if (argc != 2) {
+		DMERR("Message to snapshot received, but two args are required");
+		return 0;
+	}
+
+	if (strncmp(argv[0], "notify", 6)) {
+		DMERR("Message to snapshot received, but should be 'notify <%%>'");
+		return 0;
+	}
+
+	if (sscanf(argv[1], "%d", &snap->event_percent) != 1) {
+		DMERR("Message to snapshot received, but should be 'notify <%%>'");
+		return 0;
+	}
+
+	if ((snap->event_percent < 0) || (snap->event_percent > 99)) {
+		DMERR("Invalid notification percentage to snapshot - resetting to 0");
+		snap->event_percent = 0;
+	}
+}
+
 /*-----------------------------------------------------------------
  * Origin methods
  *---------------------------------------------------------------*/
@@ -1382,6 +1418,7 @@ static struct target_type snapshot_target = {
 	.end_io  = snapshot_end_io,
 	.resume  = snapshot_resume,
 	.status  = snapshot_status,
+	.message = snapshot_message,
 };
 
 static int __init dm_snapshot_init(void)
diff --git a/drivers/md/dm-snap.h b/drivers/md/dm-snap.h
index f07315f..15ed4c1 100644
--- a/drivers/md/dm-snap.h
+++ b/drivers/md/dm-snap.h
@@ -149,7 +149,8 @@ struct dm_snapshot {
 	chunk_t chunk_mask;
 	chunk_t chunk_shift;
 
-	/* You can't use a snapshot if this is 0 (e.g. if full) */
+	/* You can't use a snapshot if 'valid' is 0 (e.g. if full) */
+	int event_percent;
 	int valid;
 
 	/* Origin writes don't trigger exceptions until this is set */

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* RE: [linux-lvm] How to monitor and possibly autoextend snapshots
  2008-10-28 20:50     ` Jonathan Brassow
  2008-10-29 11:10       ` Marcin Kałuża
@ 2008-10-29 14:26       ` Marcin Kałuża
  2008-10-29 16:16         ` Jonathan Brassow
  1 sibling, 1 reply; 10+ messages in thread
From: Marcin Kałuża @ 2008-10-29 14:26 UTC (permalink / raw)
  To: 'LVM general discussion and development'

After initial testing I can say that it Works :) at least with
2.6.18-92.1.13.el5.centos.plus x86_64

minor variation is that I needed to use 

dmsetup wait <device> event_nr

because every time the message was signaled,
the counter increases and the default for event_nr is 0, not current. 
Where can I read something about those events and messages? (Apart from the
source code itself of course ;)  I like the way it works, but I'd like to
know how and why - mainly to know when it can fail ;)

My main concern is what could interfere with this mechanism? Are there any
kinds of events and how can I know that this event that occurred is the one
I'm waiting for. 

I googled for it, by found no useful info...

Is there a way to put it into mainstream lvm at some time (because now I'll
have to maintain my custom build kernels...)?

Are there any chances to implement snapshot autoextension mechanism, so it
doesn't require external programs?

Thanks for help
Martin

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] On
Behalf Of Jonathan Brassow
Sent: Tuesday, October 28, 2008 9:51 PM
To: LVM general discussion and development
Subject: Re: [linux-lvm] How to monitor and possibly autoextend snapshots

On Oct 27, 2008, at 7:17 AM, Marcin Ka�u�a wrote:

> Hi,
> Well, that's not much of help since after it becomes invalid I'll  
> know it
> anyway (IO error ;D), so the event is purely informational.


Well, I've thrown together a quick patch that should give you what you  
want.  (I have not compiled/tested it.)  We'd have to give it more  
thought for it to become a permanent feature (or get integrated into  
LVM), but it should be a nice little hack.  If you turn out to like  
the idea, maybe more discussion will evolve.

You'll have to compile your own kernel with the patch applied, but  
once you've done so, you should be able to do the following:

# Get snapshot to raise an event when 75% full
prompt> dmsetup message <snapshot device> 0 notify 75
prompt> dmsetup wait <snapshot device>
# Ok, now we are 75% full - let me know when 95% full
prompt> dmsetup message <snapshot device> 0 notify 95
prompt> dmsetup wait <snapshot device>

  brassow

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [linux-lvm] How to monitor and possibly autoextend snapshots
  2008-10-29 14:26       ` Marcin Kałuża
@ 2008-10-29 16:16         ` Jonathan Brassow
  2008-11-04 12:10           ` Marcin Kałuża
  0 siblings, 1 reply; 10+ messages in thread
From: Jonathan Brassow @ 2008-10-29 16:16 UTC (permalink / raw)
  To: LVM general discussion and development

Firstly, thanks for the fixes to the code.  The last one you mentioned  
(s/"s->ti->table"/"s->table"/) just depends on the kernel you are  
using.  (My patch was against upstream kernel.)

On Oct 29, 2008, at 9:26 AM, Marcin Kałuża wrote:

> minor variation is that I needed to use
>
> dmsetup wait <device> event_nr
>
> because every time the message was signaled,
> the counter increases and the default for event_nr is 0, not current.

Yup, I forgot to mention that...  Its done that way so that you can  
keep track if you have handled a particular event or not.

> Where can I read something about those events and messages? (Apart  
> from the
> source code itself of course ;)  I like the way it works, but I'd  
> like to
> know how and why - mainly to know when it can fail ;)

Yeah... there's not alot of documentation on that - except for some  
small snippets that are sprinkled in the man pages.  Let me explain  
just a little bit more.

There is inbound and outbound communication to device-mapper targets.   
The inbound (userspace -> kernel) comes in the form of the 'message'  
interface.  Every device-mapper target type (linear, stripe, mirror,  
snapshot, etc) has the option of implementing this function, but is  
not required to - in fact, most do not.  You could use the message  
interface to change device characteristics on the fly.  In our case, I  
implemented the message interface for the snapshot target - giving you  
the ability to tell a particular snapshot to raise an event when it  
gets X full.  Most targets forgo implementing this function because in  
most cases, you can specify a capability that you want when the device  
is created (through the constructor).  However, I think it our case,  
it is preferable to do it our way because we can change and reset when  
we want the notification while the device is live - rather than having  
to suspend and resume with a different constructor string.

The outbound (kernel -> userspace) communication comes in two forms -  
events and status.  Events (currently) are just a way to signal  
userspace that there is something of significance to be known.  The  
way you get that info is through status.  Events can be raised for any  
number of reasons, so you must read the status to determine if it is  
the event you are looking for.  (For example, mirrors raise events  
when the mirror becomes in-sync or when there is some sort of  
failure.  You would need to get the status to determine which it is  
and what kind of action to take.)

> My main concern is what could interfere with this mechanism? Are  
> there any
> kinds of events and how can I know that this event that occurred is  
> the one
> I'm waiting for.

You check the status.  :)

> I googled for it, by found no useful info...
>
> Is there a way to put it into mainstream lvm at some time (because  
> now I'll
> have to maintain my custom build kernels...)?

I think it's a good idea to have this capability, and it has come up  
before; so there is interest.  It can take a while for these things to  
progress, but here is how I see it playing out:

1) Get buy-in from the community (especially key players in device- 
mapper, like agk) and have them accept some version of the patch in  
this thread.  I like the patch because it doesn't try to do everything  
in the kernel - setting and resetting the thresholds is done by  
userspace.  This makes it's use very flexible.

The patch will then go upstream and then come back into other  
distribution kernels.  At this point, you will be able to implement  
your solution, but the features will not be automated.

2) Come to a community consensus as to the arguments and interface  
necessary to specify auto-response to thresholds in snapshots.  Then  
implement that in LVM.  This will take longer because the bulk of the  
work would be done here.

The patch will then go upstream and then come back into distribution  
LVM packages.

I'm not sure how long these steps will take - it depends on how  
enthusiastic people are about it.

  brassow




>
>
> Are there any chances to implement snapshot autoextension mechanism,  
> so it
> doesn't require external programs?

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: [linux-lvm] How to monitor and possibly autoextend snapshots
  2008-10-29 16:16         ` Jonathan Brassow
@ 2008-11-04 12:10           ` Marcin Kałuża
  0 siblings, 0 replies; 10+ messages in thread
From: Marcin Kałuża @ 2008-11-04 12:10 UTC (permalink / raw)
  To: 'LVM general discussion and development'



> -----Original Message-----
> From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
> On Behalf Of Jonathan Brassow
> Sent: Wednesday, October 29, 2008 5:17 PM
> To: LVM general discussion and development
> Subject: Re: [linux-lvm] How to monitor and possibly autoextend snapshots
> 
> Firstly, thanks for the fixes to the code.  The last one you mentioned
> (s/"s->ti->table"/"s->table"/) just depends on the kernel you are
> using.  (My patch was against upstream kernel.)
> 
> On Oct 29, 2008, at 9:26 AM, Marcin Ka�u�a wrote:
> 
> > minor variation is that I needed to use
> >
> > dmsetup wait <device> event_nr
> >
> > because every time the message was signaled,
> > the counter increases and the default for event_nr is 0, not current.
> 
> Yup, I forgot to mention that...  Its done that way so that you can
> keep track if you have handled a particular event or not.
> 
> > Where can I read something about those events and messages? (Apart
> > from the
> > source code itself of course ;)  I like the way it works, but I'd
> > like to
> > know how and why - mainly to know when it can fail ;)
> 
> Yeah... there's not alot of documentation on that - except for some
> small snippets that are sprinkled in the man pages.  Let me explain
> just a little bit more.
> 
> There is inbound and outbound communication to device-mapper targets.
> The inbound (userspace -> kernel) comes in the form of the 'message'
> interface.  Every device-mapper target type (linear, stripe, mirror,
> snapshot, etc) has the option of implementing this function, but is
> not required to - in fact, most do not.  You could use the message
> interface to change device characteristics on the fly.  In our case, I
> implemented the message interface for the snapshot target - giving you
> the ability to tell a particular snapshot to raise an event when it
> gets X full.  Most targets forgo implementing this function because in
> most cases, you can specify a capability that you want when the device
> is created (through the constructor).  However, I think it our case,
> it is preferable to do it our way because we can change and reset when
> we want the notification while the device is live - rather than having
> to suspend and resume with a different constructor string.
> 
> The outbound (kernel -> userspace) communication comes in two forms -
> events and status.  Events (currently) are just a way to signal
> userspace that there is something of significance to be known.  The
> way you get that info is through status.  Events can be raised for any
> number of reasons, so you must read the status to determine if it is
> the event you are looking for.  (For example, mirrors raise events
> when the mirror becomes in-sync or when there is some sort of
> failure.  You would need to get the status to determine which it is
> and what kind of action to take.)


Thanks - his clears things up a bit :)

> 
> > My main concern is what could interfere with this mechanism? Are
> > there any
> > kinds of events and how can I know that this event that occurred is
> > the one
> > I'm waiting for.
> 
> You check the status.  :)
I still didn't figure out how to get the kind of event, that happened...
dmsetup wait -v returns the same thing as "info" and "status" just gives the
size and %usage, but I can live without it :)

 
> > I googled for it, by found no useful info...
> >
> > Is there a way to put it into mainstream lvm at some time (because
> > now I'll
> > have to maintain my custom build kernels...)?
> 
> I think it's a good idea to have this capability, and it has come up
> before; so there is interest.  It can take a while for these things to
> progress, but here is how I see it playing out:
> 
> 1) Get buy-in from the community (especially key players in device-
> mapper, like agk) and have them accept some version of the patch in
> this thread.  I like the patch because it doesn't try to do everything
> in the kernel - setting and resetting the thresholds is done by
> userspace.  This makes it's use very flexible.
> 
> The patch will then go upstream and then come back into other
> distribution kernels.  At this point, you will be able to implement
> your solution, but the features will not be automated.

I never did such things, but I'll go and ask on dm-devel.

> 2) Come to a community consensus as to the arguments and interface
> necessary to specify auto-response to thresholds in snapshots.  Then
> implement that in LVM.  This will take longer because the bulk of the
> work would be done here.
> 
> The patch will then go upstream and then come back into distribution
> LVM packages.
> 
> I'm not sure how long these steps will take - it depends on how
> enthusiastic people are about it.

I've recently found some code for doing it with dmeventd, but somehow I
can't make it work with snapshots (at least not on centos 5.2/64). Again -
I'll go and ask on dm-devel about it.

Thanks for all the help
Martin

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2008-11-04 12:40 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-10-24  9:33 [linux-lvm] How to monitor and possibly autoextend snapshots marcin.kaluza
2008-10-24 16:30 ` Jonathan Brassow
2008-10-27 12:17   ` Marcin Kałuża
2008-10-28 20:50     ` Jonathan Brassow
2008-10-29 11:10       ` Marcin Kałuża
2008-10-29 14:26       ` Marcin Kałuża
2008-10-29 16:16         ` Jonathan Brassow
2008-11-04 12:10           ` Marcin Kałuża
2008-10-28  6:43 ` [linux-lvm] libdlm.h: No such file or directory Stepan Kadlec
2008-10-28  8:30   ` Christine Caulfield

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox