* merging of two file system
@ 2008-07-27 16:05 Vineet Agarwal
2008-07-27 22:52 ` Theodore Tso
2008-07-27 23:19 ` Shehjar Tikoo
0 siblings, 2 replies; 11+ messages in thread
From: Vineet Agarwal @ 2008-07-27 16:05 UTC (permalink / raw)
To: linux-ext4, listar
can we merge two file system into one in ext2/3 ?
--
>From :
Vineet Agarwal
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: merging of two file system 2008-07-27 16:05 merging of two file system Vineet Agarwal @ 2008-07-27 22:52 ` Theodore Tso 2008-07-28 18:59 ` SandeepKsinha [not found] ` <5feb302e0807281147y7362a227kb7d37d713be05296@mail.gmail.com> 2008-07-27 23:19 ` Shehjar Tikoo 1 sibling, 2 replies; 11+ messages in thread From: Theodore Tso @ 2008-07-27 22:52 UTC (permalink / raw) To: Vineet Agarwal; +Cc: linux-ext4, listar On Sun, Jul 27, 2008 at 09:35:30PM +0530, Vineet Agarwal wrote: > can we merge two file system into one in ext2/3 ? What exactly do you mean by that? - Ted ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: merging of two file system 2008-07-27 22:52 ` Theodore Tso @ 2008-07-28 18:59 ` SandeepKsinha [not found] ` <5feb302e0807281147y7362a227kb7d37d713be05296@mail.gmail.com> 1 sibling, 0 replies; 11+ messages in thread From: SandeepKsinha @ 2008-07-28 18:59 UTC (permalink / raw) To: linux-ext4 Theodore Tso wrote: > > On Sun, Jul 27, 2008 at 09:35:30PM +0530, Vineet Agarwal wrote: >> can we merge two file system into one in ext2/3 ? > >>What exactly do you mean by that? > > What I understand is that, he means to say that is there any existing mechanism or support by which we can merger two ext2/ext3 file systems on the fly. i.e. One of them would be the primary and the other would be a secondary. The secondary after merging will reside in the directory structure of the primary file system. HTH, Thanks, SandeepKsinha > - Ted > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- View this message in context: http://www.nabble.com/merging-of-two-file-system-tp18677733p18697116.html Sent from the linux-ext4 mailing list archive at Nabble.com. ^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <5feb302e0807281147y7362a227kb7d37d713be05296@mail.gmail.com>]
* Re: merging of two file system [not found] ` <5feb302e0807281147y7362a227kb7d37d713be05296@mail.gmail.com> @ 2008-07-28 19:11 ` Vineet Agarwal 2008-07-28 20:04 ` Theodore Tso 0 siblings, 1 reply; 11+ messages in thread From: Vineet Agarwal @ 2008-07-28 19:11 UTC (permalink / raw) To: rishi agrawal; +Cc: Theodore Tso, linux-ext4, listar yeah i meant that we have to separate filesystems, stored for example on /dev/hda1 and /dev/hda2, and we want to combine the data in the two filesystems into a single filesystem /dev/hda3, or combining the contents of /dev/hda2 so that it is also in /dev/hda1 so is there any mechanism exist to do so either online or offline . please help thank you ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: merging of two file system 2008-07-28 19:11 ` Vineet Agarwal @ 2008-07-28 20:04 ` Theodore Tso 2008-08-07 12:33 ` Goswin von Brederlow 0 siblings, 1 reply; 11+ messages in thread From: Theodore Tso @ 2008-07-28 20:04 UTC (permalink / raw) To: Vineet Agarwal; +Cc: rishi agrawal, linux-ext4, listar On Tue, Jul 29, 2008 at 12:41:25AM +0530, Vineet Agarwal wrote: > yeah i meant that we have to separate filesystems, > stored for example on /dev/hda1 and /dev/hda2, and we want to combine > the data in the two filesystems into a single filesystem /dev/hda3, or > combining the contents of /dev/hda2 so that it is also in /dev/hda1 > > so is there any mechanism exist to do so either online or offline . please help There is no direct way to do this. How big are the filesystems involved? The standard and easist way to do this would be to copy /dev/hda2 to another device, then if /dev/hda1 was created with on-line resizing inode, expanding /dev/hda1 to fill the space taken by /dev/hda2, and then simply use "cp" to copy the contents that had previously been in /dev/hda2 into the filesystem on /dev/hda1. In theory it would be possible to write a program would take to adjacent filesystems, and map out where the blocks would be once the two partitions were combined, and then relocate blocks to make a single filesystem. No one has done such a thing, however, for any filesytem. It is definitely not a trivial thing to do, but it is not impossible; it's not that different from some of what an off-line resize2fs operations does, albeit maybe 3-4 times more complex. The main thing is that no one has ever taken the time to do such a thing, because except for truly large filesystems, it's cheaper just to get an extra disk drive, and just copy the contents off, and then recreate the filesystem. - Ted ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: merging of two file system 2008-07-28 20:04 ` Theodore Tso @ 2008-08-07 12:33 ` Goswin von Brederlow 2008-08-08 5:47 ` Vineet Agarwal 0 siblings, 1 reply; 11+ messages in thread From: Goswin von Brederlow @ 2008-08-07 12:33 UTC (permalink / raw) To: Theodore Tso; +Cc: Vineet Agarwal, rishi agrawal, linux-ext4, listar Theodore Tso <tytso@mit.edu> writes: > On Tue, Jul 29, 2008 at 12:41:25AM +0530, Vineet Agarwal wrote: >> yeah i meant that we have to separate filesystems, >> stored for example on /dev/hda1 and /dev/hda2, and we want to combine >> the data in the two filesystems into a single filesystem /dev/hda3, or >> combining the contents of /dev/hda2 so that it is also in /dev/hda1 >> >> so is there any mechanism exist to do so either online or offline . please help > > There is no direct way to do this. > > How big are the filesystems involved? The standard and easist way to > do this would be to copy /dev/hda2 to another device, then if > /dev/hda1 was created with on-line resizing inode, expanding /dev/hda1 > to fill the space taken by /dev/hda2, and then simply use "cp" to copy > the contents that had previously been in /dev/hda2 into the filesystem > on /dev/hda1. Or if you have no extra disk but some free space you can copy as much as possible to hda1, shrink hda2, move hda2, grow hda1, repeat. At which point you really start to wish you had used LVM in the first place. > In theory it would be possible to write a program would take to > adjacent filesystems, and map out where the blocks would be once the > two partitions were combined, and then relocate blocks to make a > single filesystem. No one has done such a thing, however, for any > filesytem. It is definitely not a trivial thing to do, but it is not > impossible; it's not that different from some of what an off-line > resize2fs operations does, albeit maybe 3-4 times more complex. > > The main thing is that no one has ever taken the time to do such a > thing, because except for truly large filesystems, it's cheaper just > to get an extra disk drive, and just copy the contents off, and then > recreate the filesystem. I've started such a thing. Not specifically for merging 2 filesystem but for changing from one filesystem to another. Actually I did it even more generic by attacking the problem at the block devie layer. The method was this: 1) Convert the physical block device into a sparse device - Copy the first few blocks to a safe place - Map the first few blocks to the safe place and the rest to the physical device - Fill the old filesystem with zero (this frees the blocks from the mapping and creates empty space) 2) Create a 2nd sparse device (fully empty) 3) successivley move files from old to new and write more zeroes to old (this allocates free blocks to the 2nd device as data gets written and frees blocks from the 1st devcie as zeroes are written) 4) Destroy empty 1st device 5) Convert sparse 2nd device to physical - Move first few virtual blocks to safe place - Defragment all other blocks so virtual == physical - move blocks from safe palce to physical overwriting the sparse device metadata The safe palce is used as extra storage when the physical device has not enough blocks and to store recovery state infos. With enough free space on the FS only a few MB are needed. I used this to convert from xfs to ext3 but then run into a bug during defragmenting and had to restore from backup. As I used ext3 to restore to I had no need for this anymore so I never finished it. MfG Goswin ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: merging of two file system 2008-08-07 12:33 ` Goswin von Brederlow @ 2008-08-08 5:47 ` Vineet Agarwal 0 siblings, 0 replies; 11+ messages in thread From: Vineet Agarwal @ 2008-08-08 5:47 UTC (permalink / raw) To: Goswin von Brederlow; +Cc: Theodore Tso, rishi agrawal, linux-ext4, listar On Thu, Aug 7, 2008 at 6:03 PM, Goswin von Brederlow <goswin-v-b@web.de> wrote: > Theodore Tso <tytso@mit.edu> writes: > >> On Tue, Jul 29, 2008 at 12:41:25AM +0530, Vineet Agarwal wrote: >>> yeah i meant that we have to separate filesystems, >>> stored for example on /dev/hda1 and /dev/hda2, and we want to combine >>> the data in the two filesystems into a single filesystem /dev/hda3, or >>> combining the contents of /dev/hda2 so that it is also in /dev/hda1 >>> >>> so is there any mechanism exist to do so either online or offline . please help >> >> There is no direct way to do this. >> >> How big are the filesystems involved? The standard and easist way to >> do this would be to copy /dev/hda2 to another device, then if >> /dev/hda1 was created with on-line resizing inode, expanding /dev/hda1 >> to fill the space taken by /dev/hda2, and then simply use "cp" to copy >> the contents that had previously been in /dev/hda2 into the filesystem >> on /dev/hda1. > > Or if you have no extra disk but some free space you can copy as much > as possible to hda1, shrink hda2, move hda2, grow hda1, repeat. > > At which point you really start to wish you had used LVM in the first > place. > >> In theory it would be possible to write a program would take to >> adjacent filesystems, and map out where the blocks would be once the >> two partitions were combined, and then relocate blocks to make a >> single filesystem. No one has done such a thing, however, for any >> filesytem. It is definitely not a trivial thing to do, but it is not >> impossible; it's not that different from some of what an off-line >> resize2fs operations does, albeit maybe 3-4 times more complex. >> >> The main thing is that no one has ever taken the time to do such a >> thing, because except for truly large filesystems, it's cheaper just >> to get an extra disk drive, and just copy the contents off, and then >> recreate the filesystem. > > I've started such a thing. Not specifically for merging 2 filesystem > but for changing from one filesystem to another. Actually I did it > even more generic by attacking the problem at the block device layer. > The method was this: > > 1) Convert the physical block device into a sparse device > - Copy the first few blocks to a safe place > - Map the first few blocks to the safe place and the rest to the > physical device > - Fill the old filesystem with zero (this frees the blocks from the > mapping and creates empty space) > 2) Create a 2nd sparse device (fully empty) > 3) successivley move files from old to new and write more zeroes to > old (this allocates free blocks to the 2nd device as data gets > written and frees blocks from the 1st devcie as zeroes are written) > 4) Destroy empty 1st device > 5) Convert sparse 2nd device to physical > - Move first few virtual blocks to safe place > - Defragment all other blocks so virtual == physical > - move blocks from safe palce to physical overwriting the sparse > device metadata > > The safe palce is used as extra storage when the physical device has > not enough blocks and to store recovery state infos. With enough free > space on the FS only a few MB are needed. > > > I used this to convert from xfs to ext3 but then run into a bug during > defragmenting and had to restore from backup. As I used ext3 to > restore to I had no need for this anymore so I never finished it. > > MfG > Goswin > the concept thought is a bit similar but have not yet implemented i thought to append or copy the the superblock information from second hdd to first and then map then all or i also thought to use lvm over two fs but i don't have much knowledge about it, still working on it and can u help me to figure it out i am still working for the design if you have any idea please suggest. Thankyou -- >From : Vineet Agarwal ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: merging of two file system 2008-07-27 16:05 merging of two file system Vineet Agarwal 2008-07-27 22:52 ` Theodore Tso @ 2008-07-27 23:19 ` Shehjar Tikoo 2008-08-07 12:12 ` Goswin von Brederlow 1 sibling, 1 reply; 11+ messages in thread From: Shehjar Tikoo @ 2008-07-27 23:19 UTC (permalink / raw) To: Vineet Agarwal; +Cc: linux-ext4, listar Vineet Agarwal wrote: > can we merge two file system into one in ext2/3 ? > UnionFS might be what you're looking for. -Shehjar ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: merging of two file system 2008-07-27 23:19 ` Shehjar Tikoo @ 2008-08-07 12:12 ` Goswin von Brederlow 0 siblings, 0 replies; 11+ messages in thread From: Goswin von Brederlow @ 2008-08-07 12:12 UTC (permalink / raw) To: Shehjar Tikoo; +Cc: Vineet Agarwal, linux-ext4, listar Shehjar Tikoo <shehjart@cse.unsw.edu.au> writes: > Vineet Agarwal wrote: >> can we merge two file system into one in ext2/3 ? >> > > UnionFS might be what you're looking for. > > -Shehjar I prefer unionfs fuse. When merging directories on NFS the kernel unionFS would oops when someone changed files on the server. Verry nasty. MfG Goswin ^ permalink raw reply [flat|nested] 11+ messages in thread
* merging of two file system
@ 2008-07-27 11:18 Vineet Agarwal
2008-07-27 22:51 ` Theodore Tso
0 siblings, 1 reply; 11+ messages in thread
From: Vineet Agarwal @ 2008-07-27 11:18 UTC (permalink / raw)
To: linux-ext4
can you please help about porting the feature of file system
encapsulation on ext2
--
>From :
Vineet Agarwal
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: merging of two file system 2008-07-27 11:18 Vineet Agarwal @ 2008-07-27 22:51 ` Theodore Tso 0 siblings, 0 replies; 11+ messages in thread From: Theodore Tso @ 2008-07-27 22:51 UTC (permalink / raw) To: Vineet Agarwal; +Cc: linux-ext4 On Sun, Jul 27, 2008 at 04:48:47PM +0530, Vineet Agarwal wrote: > can you please help about porting the feature of file system > encapsulation on ext2 I'm not sure what you mean by that. Regards, - Ted ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2008-08-08 5:47 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-07-27 16:05 merging of two file system Vineet Agarwal
2008-07-27 22:52 ` Theodore Tso
2008-07-28 18:59 ` SandeepKsinha
[not found] ` <5feb302e0807281147y7362a227kb7d37d713be05296@mail.gmail.com>
2008-07-28 19:11 ` Vineet Agarwal
2008-07-28 20:04 ` Theodore Tso
2008-08-07 12:33 ` Goswin von Brederlow
2008-08-08 5:47 ` Vineet Agarwal
2008-07-27 23:19 ` Shehjar Tikoo
2008-08-07 12:12 ` Goswin von Brederlow
-- strict thread matches above, loose matches on Subject: below --
2008-07-27 11:18 Vineet Agarwal
2008-07-27 22:51 ` Theodore Tso
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).