RE: [linux-lvm] read-only media in a LVM?

RE: [linux-lvm] read-only media in a LVM?

[Sean P. Kane]

Ok, let me clarify a bit. I mis-spoke, when I spoke of memory I meant
ramdisk (fs). This is basically what I was actually visualizing in my
head:

1) Boot cdrom into a Linux instance.
2) Mount a read-write filesystem into memory (this is why I asked the
size issue, obviously coping the whole cdrom to memory wouldn't be
practical)
3) Create a snapshot (I'm am really new to lvm, so I'm still learning
what all this means), which would be a small reference table of some
sort, plus 32k blocks of changed data.
4)This would then allow "changes" to the cdrom while running in that
session until the ram disk filled up? Correct?
5) If a file is created then deleted in an environment like this will
the space it was using in the snapshot fs be released?

The use I have in mind for this, doesn't need a ton of additional files
added, after boot-up but would need a lot of temp files created and then
cleaned up. 

What are the basic commands I would need to achieve this? I've read the
LVM howto (doesn't cover LVM2, that I saw) and still not quite sure.

Thanks,
Sean


------------------------------------------------------------------------
Sean P. Kane
spkane@genomatica.com
Lead Infrastructure Architect
Genomatica, Inc.
------------------------------------------------------------------------
"When we destroy something man has created we call it vandalism......
 When we destroy something that Gaia has created we call it progress."


-----Original Message-----
From: Wolfgang Weisselberg [mailto:uzx87lvfmukwc001@sneakemail.com] 
Sent: Monday, June 23, 2003 14:23
To: linux-lvm@sistina.com
Subject: Re: [linux-lvm] read-only media in a LVM?


Sean P. Kane wrote 36 lines:

> Does a snapshot create a copy of all the files in memory?

Nope.  And not in memory either.

> Just the files that changed?

Nope, LVM is filesystem-agnostic, and that would have to change that.
More code, more errors, less usability.

Additionally, what would you do with metadata, like used filesystem
blocks and timestamps and filenames?

Or with a 1 TB file where just 1 bit was changed?

> Or just the file blocks or contents (i.e. diffs) that have changed?

Blocks of (default) 32Kb of raw disk.  They are written to the snapshot.
However, the tables of which blocks were written are kept in RAM, so a
read request on the snapshot redirects only the unchanged blocks to the
original (HD/CD) and gets the changed blocks out of the snapshot
storage.

> So, would the snapshot have to be the same size as the cdrom or could 
> it be smaller?

MUCH smaller. Unless you insist on changing something in every 32KB
block of the CDrom.

-Wolfgang

_______________________________________________
linux-lvm mailing list
linux-lvm@sistina.com
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] read-only media in a LVM?

[Kevin P. Fleming]

Sean P. Kane wrote:

> Ok, let me clarify a bit. I mis-spoke, when I spoke of memory I meant
> ramdisk (fs). This is basically what I was actually visualizing in my
> head:
> 
> 1) Boot cdrom into a Linux instance.
> 2) Mount a read-write filesystem into memory (this is why I asked the
> size issue, obviously coping the whole cdrom to memory wouldn't be
> practical)
> 3) Create a snapshot (I'm am really new to lvm, so I'm still learning
> what all this means), which would be a small reference table of some
> sort, plus 32k blocks of changed data.
> 4)This would then allow "changes" to the cdrom while running in that
> session until the ram disk filled up? Correct?
> 5) If a file is created then deleted in an environment like this will
> the space it was using in the snapshot fs be released?
> 

This is not really an application for LVM snapshots at all. Snapshots would 
continue to use more and more memory if those files are created and removed, 
unless the underlying filesystem is very intelligent about space allocation 
(which the iso9660 filesystem is very likely not).

This is really an application for a "union" mount. This could also be done with 
a "bind" mount, if the files that need changing can be isolated to a small 
number of directories. If they are spread all over the filesystem, or their 
locations are not known ahead of time, a "union" mount would do what you need, 
overlaying a tmpfs ram-based filesystem on top of the CD-ROM filesystem.

RE: [linux-lvm] read-only media in a LVM?

[Sean P. Kane]

I'd love to have a unionfs, but my understanding is that no such beast
exists for Linux 2.4 currently and won't likely appear until after the
2.6 release. If this is not the case, I would love to hear about it. Is
there some decent (although maybe beta) support for a unionfs in the 2.5
kernels? I am using the LVM2 backport to 2.4 currently to test the
device-mapper approach out in the meantime, but if there is a better way
to do it, I'd love to hear about it.

Thanks,
Sean


------------------------------------------------------------------------
Sean P. Kane
spkane@genomatica.com
Lead Infrastructure Architect
Genomatica, Inc.
------------------------------------------------------------------------
"When we destroy something man has created we call it vandalism......
 When we destroy something that Gaia has created we call it progress."


-----Original Message-----
From: Kevin P. Fleming [mailto:kpfleming@cox.net] 
Sent: Monday, June 23, 2003 15:41
To: linux-lvm@sistina.com
Subject: Re: [linux-lvm] read-only media in a LVM?


Sean P. Kane wrote:

> Ok, let me clarify a bit. I mis-spoke, when I spoke of memory I meant 
> ramdisk (fs). This is basically what I was actually visualizing in my
> head:
> 
> 1) Boot cdrom into a Linux instance.
> 2) Mount a read-write filesystem into memory (this is why I asked the 
> size issue, obviously coping the whole cdrom to memory wouldn't be
> practical)
> 3) Create a snapshot (I'm am really new to lvm, so I'm still learning 
> what all this means), which would be a small reference table of some 
> sort, plus 32k blocks of changed data. 4)This would then allow 
> "changes" to the cdrom while running in that session until the ram 
> disk filled up? Correct?
> 5) If a file is created then deleted in an environment like this will 
> the space it was using in the snapshot fs be released?
> 

This is not really an application for LVM snapshots at all. Snapshots
would 
continue to use more and more memory if those files are created and
removed, 
unless the underlying filesystem is very intelligent about space
allocation 
(which the iso9660 filesystem is very likely not).

This is really an application for a "union" mount. This could also be
done with 
a "bind" mount, if the files that need changing can be isolated to a
small 
number of directories. If they are spread all over the filesystem, or
their 
locations are not known ahead of time, a "union" mount would do what you
need, 
overlaying a tmpfs ram-based filesystem on top of the CD-ROM filesystem.


_______________________________________________
linux-lvm mailing list
linux-lvm@sistina.com
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] read-only media in a LVM?

[Wolfgang Weisselberg]

Sean P. Kane wrote 78 lines:

> I'd love to have a unionfs, but my understanding is that no such beast
> exists for Linux 2.4 currently

Try looking at 
    http://plasticfs.sourceforge.net/plasticfs.html

Plasticfs-viewpath is basically a unionof all filesystems in
the viewpath, changes happen on the first one and a .whiteout
remembers what you deleted.  It does *not* work for root
(security!!!  You do not want root to see /tmp/login instead of
/bin/login!).  Remember to set the environment variables, e.g.

    LD_PRELOAD=/usr/local/lib/libplasticfs.so
    PLASTICFS="viewpath /ramdisk /cdrom1 /cdrom2"
    /bin/bash   # or whatever program you want.

Since Plasticfs works on file-level, you should be able to mix
and match filesystems as you like.  Note however that --- since
we just overload common FS access methods --- some things might
not work.

I have it in production use (for Aegis, not for cdroms).

-Wolfgang

Re: [linux-lvm] read-only media in a LVM?

[Kevin P. Fleming]

Sean P. Kane wrote:
> I'd love to have a unionfs, but my understanding is that no such beast
> exists for Linux 2.4 currently and won't likely appear until after the
> 2.6 release. If this is not the case, I would love to hear about it. Is
> there some decent (although maybe beta) support for a unionfs in the 2.5
> kernels? I am using the LVM2 backport to 2.4 currently to test the
> device-mapper approach out in the meantime, but if there is a better way
> to do it, I'd love to hear about it.
> 

No, this won't appear for 2.4 kernels at any time, I'd wager. However 
2.4 kernels support mount --bind just fine, so that may be an option 
for you if the writable directories in your filesystem can be 
predetermined.

RE: [linux-lvm] read-only media in a LVM?

[Sean P. Kane]

Wolfgang, can I get your email address so that I can ask you a few
questions about this off-line?

Thanks,
Sean


------------------------------------------------------------------------
Sean P. Kane
spkane@genomatica.com
Lead Infrastructure Architect
Genomatica, Inc.
------------------------------------------------------------------------
"When we destroy something man has created we call it vandalism......
 When we destroy something that Gaia has created we call it progress."


-----Original Message-----
From: Wolfgang Weisselberg [mailto:uzx87lvfmukwc001@sneakemail.com] 
Sent: Monday, June 23, 2003 17:35
To: linux-lvm@sistina.com
Subject: Re: [linux-lvm] read-only media in a LVM?


Sean P. Kane wrote 78 lines:

> I'd love to have a unionfs, but my understanding is that no such beast

> exists for Linux 2.4 currently

Try looking at 
    http://plasticfs.sourceforge.net/plasticfs.html

Plasticfs-viewpath is basically a unionof all filesystems in the
viewpath, changes happen on the first one and a .whiteout remembers what
you deleted.  It does *not* work for root (security!!!  You do not want
root to see /tmp/login instead of /bin/login!).  Remember to set the
environment variables, e.g.

    LD_PRELOAD=/usr/local/lib/libplasticfs.so
    PLASTICFS="viewpath /ramdisk /cdrom1 /cdrom2"
    /bin/bash   # or whatever program you want.

Since Plasticfs works on file-level, you should be able to mix and match
filesystems as you like.  Note however that --- since we just overload
common FS access methods --- some things might not work.

I have it in production use (for Aegis, not for cdroms).

-Wolfgang

_______________________________________________
linux-lvm mailing list
linux-lvm@sistina.com
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

RE: [linux-lvm] read-only media in a LVM?

[Sean P. Kane]

Thanks for all the useful thoughts and pointers. I'll give them a try
and see what I can manage to pull together.

Sean

------------------------------------------------------------------------
Sean P. Kane
spkane@genomatica.com
Lead Infrastructure Architect
Genomatica, Inc.
------------------------------------------------------------------------
"When we destroy something man has created we call it vandalism......
 When we destroy something that Gaia has created we call it progress."


-----Original Message-----
From: Wolfgang Weisselberg [mailto:uzx87lvfmukwc001@sneakemail.com] 
Sent: Monday, June 23, 2003 17:35
To: linux-lvm@sistina.com
Subject: Re: [linux-lvm] read-only media in a LVM?


Sean P. Kane wrote 78 lines:

> I'd love to have a unionfs, but my understanding is that no such beast

> exists for Linux 2.4 currently

Try looking at 
    http://plasticfs.sourceforge.net/plasticfs.html

Plasticfs-viewpath is basically a unionof all filesystems in the
viewpath, changes happen on the first one and a .whiteout remembers what
you deleted.  It does *not* work for root (security!!!  You do not want
root to see /tmp/login instead of /bin/login!).  Remember to set the
environment variables, e.g.

    LD_PRELOAD=/usr/local/lib/libplasticfs.so
    PLASTICFS="viewpath /ramdisk /cdrom1 /cdrom2"
    /bin/bash   # or whatever program you want.

Since Plasticfs works on file-level, you should be able to mix and match
filesystems as you like.  Note however that --- since we just overload
common FS access methods --- some things might not work.

I have it in production use (for Aegis, not for cdroms).

-Wolfgang

_______________________________________________
linux-lvm mailing list
linux-lvm@sistina.com
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

RE: [linux-lvm] read-only media in a LVM?

[Sean P. Kane]

Does a snapshot create a copy of all the files in memory? Just the files
that changed? Or just the file blocks or contents (i.e. diffs) that have
changed? So, would the snapshot have to be the same size as the cdrom or
could it be smaller?

Thanks,
Sean


------------------------------------------------------------------------
Sean P. Kane
spkane@genomatica.com
Lead Infrastructure Architect
Genomatica, Inc.
------------------------------------------------------------------------
"When we destroy something man has created we call it vandalism......
 When we destroy something that Gaia has created we call it progress."


-----Original Message-----
On Fri, Jun 13, 2003 at 04:55:40PM -0700, Sean P. Kane wrote:
> I saw this email below and was wondering if this is supported in both
dm
> and the LVM2 tools now.

The tools still don't support this, so you'll have to use dmsetup to
create a writeable snapshot of the CD.  Also be aware that you'll need
to have a suitable fs on the CD, most CD filesystems are read only.

- Joe

Re: [linux-lvm] read-only media in a LVM?

[Wolfgang Weisselberg]

Sean P. Kane wrote 36 lines:

> Does a snapshot create a copy of all the files in memory?

Nope.  And not in memory either.

> Just the files that changed?

Nope, LVM is filesystem-agnostic, and that would have to change
that.  More code, more errors, less usability.

Additionally, what would you do with metadata, like used
filesystem blocks and timestamps and filenames?

Or with a 1 TB file where just 1 bit was changed?

> Or just the file blocks or contents (i.e. diffs) that have
> changed?

Blocks of (default) 32Kb of raw disk.  They are written to the
snapshot.  However, the tables of which blocks were written are
kept in RAM, so a read request on the snapshot redirects only the
unchanged blocks to the original (HD/CD) and gets the changed
blocks out of the snapshot storage.

> So, would the snapshot have to be the same size as the cdrom or
> could it be smaller?

MUCH smaller. Unless you insist on changing something in every
32KB block of the CDrom.

-Wolfgang

RE: [linux-lvm] read-only media in a LVM?

[Sean P. Kane]

[-- Attachment #1: Type: text/plain, Size: 2612 bytes --]

I saw this email below and was wondering if this is supported in both dm
and the LVM2 tools now. I am trying to setup a bootable cd, that can be
written to during that session and this would be a god send, since as
the last person points out there really isn't a mature way to do this in
the 2.4/2.5 kernels. 2.6 is suppose to have a unionfs sometime after
release, but this is an immediate project.
 
Thanks,
Sean
 
------------------------------------------------------------------------
Sean P. Kane
spkane@genomatica.com
Lead Infrastructure Architect
Genomatica, Inc. <http://www.genomatica.com/> 
------------------------------------------------------------------------
"When we destroy something man has created we call it vandalism......
 When we destroy something that Gaia
<http://dictionary.reference.com/search?q=gaia>  has created we call it
progress."

 

[linux-lvm] read-only media in a LVM? 
Joe Thornber linux-lvm@sistina.com <mailto:linux-lvm%40sistina.com> 
Tue May 28 03:44:01 2002 
 
Previous message: [linux-lvm] read-only media in a LVM?
<https://lists.sistina.com/pipermail/linux-lvm/2002-May/011529.html> 
Next message: [linux-lvm] errors compiling on redhat 7.3
<https://lists.sistina.com/pipermail/linux-lvm/2002-May/011528.html> 
Messages sorted by: [ date ]
<https://lists.sistina.com/pipermail/linux-lvm/2002-May/date.html#11530>
[ thread ]
<https://lists.sistina.com/pipermail/linux-lvm/2002-May/thread.html#1153
0>  [ subject ]
<https://lists.sistina.com/pipermail/linux-lvm/2002-May/subject.html#115
30>  [ author ]
<https://lists.sistina.com/pipermail/linux-lvm/2002-May/author.html#1153
0>  
  _____  

On Mon, May 27, 2002 at 04:37:49PM -0700, Ami Fischman wrote:
> Hi there,
> I am wondering whether it is possible to have a read-only fs (CDROM,
image,
> etc) as part of an LVM-managed volume. I see references to COW
> (copy-on-write, I'm hoping :)) in the kernel drivers/md/ files, but
it's not
> at all clear to me whether or not the current drivers can be used to
> implement a "translucent" or "inheriting" file system (one where a
base fs
> is given, on r-o media, and changes to it get recorded on another
device,
> presumabely a r-w fs). Please let me know whether this is currently
> possible, being planned for the future, or if I'm making no sense
above &
> you want me to clarify :). For a not-quite-mature example of what I'm
> talking about, see translucency.sf.net.
 
In fact you can do this now if you use just device-mapper/dmsetup :)
It's just the LVM2 tools that don't support it yet.- Joe 
 

[-- Attachment #2: Type: text/html, Size: 5733 bytes --]

Re: [linux-lvm] read-only media in a LVM?

[Joe Thornber]

On Fri, Jun 13, 2003 at 04:55:40PM -0700, Sean P. Kane wrote:
> I saw this email below and was wondering if this is supported in both dm
> and the LVM2 tools now.

The tools still don't support this, so you'll have to use dmsetup to
create a writeable snapshot of the CD.  Also be aware that you'll need
to have a suitable fs on the CD, most CD filesystems are read only.

- Joe

[linux-lvm] read-only media in a LVM?

[Ami Fischman]

Hi there,
I am wondering whether it is possible to have a read-only fs (CDROM, image,
etc) as part of an LVM-managed volume.  I see references to COW
(copy-on-write, I'm hoping :)) in the kernel drivers/md/ files, but it's not
at all clear to me whether or not the current drivers can be used to
implement a "translucent" or "inheriting" file system (one where a base fs
is given, on r-o media, and changes to it get recorded on another device,
presumabely a r-w fs).  Please let me know whether this is currently
possible, being planned for the future, or if I'm making no sense above &
you want me to clarify :).  For a not-quite-mature example of what I'm
talking about, see translucency.sf.net.

Thanks,
-- 
  Ami Fischman, Ph.D.
  fischman@bpnetworks.com
  http://www.bpnetworks.com/

Re: [linux-lvm] read-only media in a LVM?

[Joe Thornber]

On Mon, May 27, 2002 at 04:37:49PM -0700, Ami Fischman wrote:
> Hi there,
> I am wondering whether it is possible to have a read-only fs (CDROM, image,
> etc) as part of an LVM-managed volume.  I see references to COW
> (copy-on-write, I'm hoping :)) in the kernel drivers/md/ files, but it's not
> at all clear to me whether or not the current drivers can be used to
> implement a "translucent" or "inheriting" file system (one where a base fs
> is given, on r-o media, and changes to it get recorded on another device,
> presumabely a r-w fs).  Please let me know whether this is currently
> possible, being planned for the future, or if I'm making no sense above &
> you want me to clarify :).  For a not-quite-mature example of what I'm
> talking about, see translucency.sf.net.

LVM2 already contains writeable snapshots, and we are currently
working on the other thing that you will need which is the ability to
use disks as PVs without having any LVM metadata written to them.
Check back at the end of June, we may have done it by then.

- Joe

Re: [linux-lvm] read-only media in a LVM?

[Joe Thornber]

On Mon, May 27, 2002 at 04:37:49PM -0700, Ami Fischman wrote:
> Hi there,
> I am wondering whether it is possible to have a read-only fs (CDROM, image,
> etc) as part of an LVM-managed volume.  I see references to COW
> (copy-on-write, I'm hoping :)) in the kernel drivers/md/ files, but it's not
> at all clear to me whether or not the current drivers can be used to
> implement a "translucent" or "inheriting" file system (one where a base fs
> is given, on r-o media, and changes to it get recorded on another device,
> presumabely a r-w fs).  Please let me know whether this is currently
> possible, being planned for the future, or if I'm making no sense above &
> you want me to clarify :).  For a not-quite-mature example of what I'm
> talking about, see translucency.sf.net.

In fact you can do this now if you use just device-mapper/dmsetup :)
It's just the LVM2 tools that don't support it yet.

- Joe