* clone of filesystem across network preserving ionodes @ 2014-11-03 20:35 Meij, Henk 2014-11-03 20:39 ` Eric Sandeen 2014-11-04 0:05 ` Peter Grandi 0 siblings, 2 replies; 6+ messages in thread From: Meij, Henk @ 2014-11-03 20:35 UTC (permalink / raw) To: xfs@oss.sgi.com [-- Attachment #1.1: Type: text/plain, Size: 368 bytes --] What is the best practice way to get the contents from server1 to server2 identical disk array partitions and preserve the ionode information using xfs tools or non-xfs tools? xfsdump/xfsrestore appear to not preserve this or I can not find the obvious setting. xfs_copy would, per documentation, but can I get from server1 to server2? -Henk [-- Attachment #1.2: Type: text/html, Size: 774 bytes --] [-- Attachment #2: Type: text/plain, Size: 121 bytes --] _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: clone of filesystem across network preserving ionodes 2014-11-03 20:35 clone of filesystem across network preserving ionodes Meij, Henk @ 2014-11-03 20:39 ` Eric Sandeen 2014-11-03 21:10 ` Meij, Henk 2014-11-04 0:05 ` Peter Grandi 1 sibling, 1 reply; 6+ messages in thread From: Eric Sandeen @ 2014-11-03 20:39 UTC (permalink / raw) To: Meij, Henk, xfs@oss.sgi.com On 11/3/14 2:35 PM, Meij, Henk wrote: > What is the best practice way to get the contents from server1 to > server2 identical disk array partitions and preserve the ionode > information using xfs tools or non-xfs tools? Just FWIW: inode, not ionode. :) > xfsdump/xfsrestore appear to not preserve this or I can not find the > obvious setting. > xfs_copy would, per documentation, but can I get from server1 to > server2? xfs_copy makes an identical image, yes. You could xfs_copy from disk to a file on server1, transfer the file, then xfs_copy from file to disk on server2. -Eric _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: clone of filesystem across network preserving ionodes 2014-11-03 20:39 ` Eric Sandeen @ 2014-11-03 21:10 ` Meij, Henk 2014-11-03 21:56 ` Dave Chinner 0 siblings, 1 reply; 6+ messages in thread From: Meij, Henk @ 2014-11-03 21:10 UTC (permalink / raw) To: Eric Sandeen, xfs@oss.sgi.com inodes, yes, yes. I tried xfs_copy but ran into a problem. server1 and server2 each have 4x28T partitions and on server1 writing the file to one empty partition mounted never finishes, hangs at 90%...presumably because it runs out of space (file size == sdbx). there is no "skip empty inodes" option and perhaps there can't be...and I have nothing larger. I guess I could attempt making sdb4 slightly larger by reducing sdb1-3. Thanks, -Henk ________________________________________ From: Eric Sandeen [sandeen@sandeen.net] Sent: Monday, November 03, 2014 3:39 PM To: Meij, Henk; xfs@oss.sgi.com Subject: Re: clone of filesystem across network preserving ionodes On 11/3/14 2:35 PM, Meij, Henk wrote: > What is the best practice way to get the contents from server1 to > server2 identical disk array partitions and preserve the ionode > information using xfs tools or non-xfs tools? Just FWIW: inode, not ionode. :) > xfsdump/xfsrestore appear to not preserve this or I can not find the > obvious setting. > xfs_copy would, per documentation, but can I get from server1 to > server2? xfs_copy makes an identical image, yes. You could xfs_copy from disk to a file on server1, transfer the file, then xfs_copy from file to disk on server2. -Eric _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: clone of filesystem across network preserving ionodes 2014-11-03 21:10 ` Meij, Henk @ 2014-11-03 21:56 ` Dave Chinner 0 siblings, 0 replies; 6+ messages in thread From: Dave Chinner @ 2014-11-03 21:56 UTC (permalink / raw) To: Meij, Henk; +Cc: Eric Sandeen, xfs@oss.sgi.com On Mon, Nov 03, 2014 at 09:10:10PM +0000, Meij, Henk wrote: > inodes, yes, yes. I tried xfs_copy but ran into a problem. > server1 and server2 each have 4x28T partitions and on server1 > writing the file to one empty partition mounted never finishes, > hangs at 90%...presumably because it runs out of space (file size > == sdbx). Sure, but it's a sparse copy. It only copies the allocated blocks in the filesystem, so the actual space required is the used space int eh filesystem (i.e. what df reports as used). > there is no "skip empty inodes" option and perhaps there > can't be... Of course not - you're wanting identical inode numbers on either end, so your only option is and identical copy. Otherwise you'd use xfsdump/xfsrestore to skip empty inodes... > and I have nothing larger. I guess I could attempt > making sdb4 slightly larger by reducing sdb1-3. Exactly why do you need an *identical* copy of 28TB filesystems? What is the problem with inode numbers being different? And that begs the question: if you need the filesystems to be completely identical yet exist on separate systems, then why aren't you using a block layer construct designed for such operation (e.g. drdb)? Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: clone of filesystem across network preserving ionodes 2014-11-03 20:35 clone of filesystem across network preserving ionodes Meij, Henk 2014-11-03 20:39 ` Eric Sandeen @ 2014-11-04 0:05 ` Peter Grandi 2014-11-04 14:16 ` Meij, Henk 1 sibling, 1 reply; 6+ messages in thread From: Peter Grandi @ 2014-11-04 0:05 UTC (permalink / raw) To: Linux fs XFS > What is the best practice way to get the contents from > server1 to server2 identical disk array partitions Here you do not say that «get the contents» has to be over the network, but the 'Subject:' line seems to indicate that. > and preserve the ionode information using xfs tools or non-xfs > tools? xfsdump/xfsrestore appear to not preserve this or I can > not find the obvious setting. If «preserve the ionode information» means "keep the same inumbers for each inode" that seems to me fairly pointless in almost every situation (except perhaps for forensic purposes, and that opens a whole series of other issues), so there is no «best practice way» as there is little or no practice, and management speak won't change that. More clarity in explaining what you mean and/or what you want to achieve would be better. If you want for whatever reason to make a byte-by-byte copy from block device X (potentially mounted on M) on host S1 to the corresponding one of «identical disk array partitions» Y on host S2, which also preserves the inumbers, something fairly common is to use 'ssh' as a network pipe (run on host S1): M="`grep \"^$X\" /proc/mounts | cut -d' ' -f2`" case "$M" in ?*) xfs_freeze -f `"$M";; esac sudo sh -c "lzop < '$X'" | dd bs=64k iflag=fullblock \ | ssh -oCipher=arcfour -oCompression=no "$S2" \ "dd bs=64k iflag=fullblock | sudo sh -c "\'"lzop -d > '$Y'"\'"" case "$M" in ?*) xfs_freeze -u `"$M";; esac If 'xfs_freeze' is suitable, and if no interruptions happen; if they happen put reasonable "seek" and "skip" parameters on 'dd' and rerun. If the target block device is bigger than the source block device, one can use 'xfs_growfs' after the byte-by-byte copy (growing filetrees has downsides though). Unless by «preserve the ionode information» or «identical disk array partitions» you mean something different from what they appear to mean to me. The above just copied for me a 88GiB block device in 1624s between two ordinary home PCs. One could replace 'ssh' with 'nc' or similar network piping commands for higher speed (at some cost). I also tried to interpolate 'xfs_copy -d -b "$X" /dev/stdout | lzop ...' but 'xfs_copy' won't do that, not entirely unexpectedly. If by «preserve the ionode information» you simply mean copy all file attributes but without keeping the inumber (except for 'atime', but is usually disabled anyhow, and there is a patch to restore it), if the source filetree is mounted at M on S1 and the target is mounted at N (you may opt to mount with 'nobarrier') on S2, a fairly common way is to use RSYNC with various preserving options (run on host S1): sudo rsync -i -axAHX --del -z \ -e 'ssh -oCipher=arcfour -oCompression=no -l root' \ "$M"/ "$S2":"$N"/ The following is another common way that has slightly different effects on 'atime' and 'ctime' (requires a patched GNU 'tar' or be replaced with 'star') by using 'ssh' combines with 'tar': sudo sh -c "cd '$M' && tar -c --one --selinux --acls --xattrs -f - ." \ | lzop | dd bs=64k iflag=fullblock \ | ssh -oCipher=arcfour -oCompression=no "$S2" \ "dd bs=64k iflag=fullblock | lzop -d \ | sudo sh -c 'cd \"$N\" && tar -xv --preserve-p --atime-p -f -'" > xfs_copy would, per documentation, Probably 'xfs_copy' is only necessary in the case where you really want to preserve inums, and the source block device is larger than the target block device (and obviously the size of the content is smaller than the size of the target block device). > but can I get from server1 to server2? That probably requires finding a target medium (file or block device) big enough and that can be shared (or transported) between S1 and S2. This could involve a filetree on S2 that is exported (for example by NFSv4) to S1 and large enough to hold the target file(s) of 'xfs_copy'. PS Whichever way is chosen to the copy, a second run with RSYNC options '-n -c' might help verify that corruption did not happens during the copy. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: clone of filesystem across network preserving ionodes 2014-11-04 0:05 ` Peter Grandi @ 2014-11-04 14:16 ` Meij, Henk 0 siblings, 0 replies; 6+ messages in thread From: Meij, Henk @ 2014-11-04 14:16 UTC (permalink / raw) To: Peter Grandi, Linux fs XFS Thank you list, I think the problem is in my work flow, and in my head. I will be testing some suggestions and will report back. To answer some of the questions: S1 and S2 are identical research storage units (112TB each) partitioned in 4 28TB partitions, with duplicate data for fail over purposes (exceeds out enterprise backup capacity, cheaper this way). DRBD is where we started. v8.4.5 was tested but could not deliver rates larger than 40MB/s, despite the lists excellent suggestions. We're unsure why; we ran w/wo BBU, changed RW policies, configured 2 global hot spares into raid 0, even made ram disks, multiple drbd devices, but the storage unit (xfs on raid 60) never obtained the "want" speeds of 100MB/s. rsync tests had no problems reaching 112MB/s and slightly better nic-to-nic. First initialization/re-sync times for DRBD would be 30 days unless fixed. So we're building our own. Pulling data from multiple locations to seed S1, then use rsync or xfsdump to seed S2. Manual fail over of service IP with scripts. Nightly incrementals. We'd like to run daily, weekly, monthly snapshots on S2. We do that with the --link-dest=/path/to/d.1 /path/to/source /path/to/d.0 ... now the first daily snapshot (d.0) will be as large as the source. And that's what I like to avoid (having 3 copies) so I need to change my thinking and present the S2 copy to --link-dest as the latest backup. -Henk ________________________________________ From: xfs-bounces@oss.sgi.com [xfs-bounces@oss.sgi.com] on behalf of Peter Grandi [pg@xfs.list.sabi.co.UK] Sent: Monday, November 03, 2014 7:05 PM To: Linux fs XFS Subject: Re: clone of filesystem across network preserving ionodes > What is the best practice way to get the contents from > server1 to server2 identical disk array partitions Here you do not say that «get the contents» has to be over the network, but the 'Subject:' line seems to indicate that. > and preserve the ionode information using xfs tools or non-xfs > tools? xfsdump/xfsrestore appear to not preserve this or I can > not find the obvious setting. If «preserve the ionode information» means "keep the same inumbers for each inode" that seems to me fairly pointless in almost every situation (except perhaps for forensic purposes, and that opens a whole series of other issues), so there is no «best practice way» as there is little or no practice, and management speak won't change that. More clarity in explaining what you mean and/or what you want to achieve would be better. If you want for whatever reason to make a byte-by-byte copy from block device X (potentially mounted on M) on host S1 to the corresponding one of «identical disk array partitions» Y on host S2, which also preserves the inumbers, something fairly common is to use 'ssh' as a network pipe (run on host S1): M="`grep \"^$X\" /proc/mounts | cut -d' ' -f2`" case "$M" in ?*) xfs_freeze -f `"$M";; esac sudo sh -c "lzop < '$X'" | dd bs=64k iflag=fullblock \ | ssh -oCipher=arcfour -oCompression=no "$S2" \ "dd bs=64k iflag=fullblock | sudo sh -c "\'"lzop -d > '$Y'"\'"" case "$M" in ?*) xfs_freeze -u `"$M";; esac If 'xfs_freeze' is suitable, and if no interruptions happen; if they happen put reasonable "seek" and "skip" parameters on 'dd' and rerun. If the target block device is bigger than the source block device, one can use 'xfs_growfs' after the byte-by-byte copy (growing filetrees has downsides though). Unless by «preserve the ionode information» or «identical disk array partitions» you mean something different from what they appear to mean to me. The above just copied for me a 88GiB block device in 1624s between two ordinary home PCs. One could replace 'ssh' with 'nc' or similar network piping commands for higher speed (at some cost). I also tried to interpolate 'xfs_copy -d -b "$X" /dev/stdout | lzop ...' but 'xfs_copy' won't do that, not entirely unexpectedly. If by «preserve the ionode information» you simply mean copy all file attributes but without keeping the inumber (except for 'atime', but is usually disabled anyhow, and there is a patch to restore it), if the source filetree is mounted at M on S1 and the target is mounted at N (you may opt to mount with 'nobarrier') on S2, a fairly common way is to use RSYNC with various preserving options (run on host S1): sudo rsync -i -axAHX --del -z \ -e 'ssh -oCipher=arcfour -oCompression=no -l root' \ "$M"/ "$S2":"$N"/ The following is another common way that has slightly different effects on 'atime' and 'ctime' (requires a patched GNU 'tar' or be replaced with 'star') by using 'ssh' combines with 'tar': sudo sh -c "cd '$M' && tar -c --one --selinux --acls --xattrs -f - ." \ | lzop | dd bs=64k iflag=fullblock \ | ssh -oCipher=arcfour -oCompression=no "$S2" \ "dd bs=64k iflag=fullblock | lzop -d \ | sudo sh -c 'cd \"$N\" && tar -xv --preserve-p --atime-p -f -'" > xfs_copy would, per documentation, Probably 'xfs_copy' is only necessary in the case where you really want to preserve inums, and the source block device is larger than the target block device (and obviously the size of the content is smaller than the size of the target block device). > but can I get from server1 to server2? That probably requires finding a target medium (file or block device) big enough and that can be shared (or transported) between S1 and S2. This could involve a filetree on S2 that is exported (for example by NFSv4) to S1 and large enough to hold the target file(s) of 'xfs_copy'. PS Whichever way is chosen to the copy, a second run with RSYNC options '-n -c' might help verify that corruption did not happens during the copy. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-11-04 14:17 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-11-03 20:35 clone of filesystem across network preserving ionodes Meij, Henk 2014-11-03 20:39 ` Eric Sandeen 2014-11-03 21:10 ` Meij, Henk 2014-11-03 21:56 ` Dave Chinner 2014-11-04 0:05 ` Peter Grandi 2014-11-04 14:16 ` Meij, Henk
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox