writing disk image to file

writing disk image to file

[James Miller]

I've got a small disk whose contents I'd like to save as an image file, in
case I ever want to write it back to the disk it came from.  This is a 4MB
flash disk.  From what I gather in my Linux reading, I should do something
like the following: hook this tiny IDE disk up as, say /dev/hdb and the do
something like "dd if=/dev/hdb of=dev-hdb-image.img" ? ? ? (the question
marks outside the quotes indicates things I'm not sure about).  This will
write an image file to, I suppose the current dir - correct?  Is the
output file's name somewhat a matter of indifference, as it seems to me?
E.g., wouldn't necessarily require any extension (might be called simply
"dev-hdb-image"), though an extension might help me keep track of just
what sort of file it is.  And, finally, the question marks: I see
sometimes "bs=512" or some such following the proposed file name which, as
I get it, refers to block size.  I'm not really clear on block sizes and
what they do, but I have run accross information indicating that, in the
case of an image file, it's not too important and can be left out.  Can
anyone illumine my benightedness on this matter?

Thanks, James

PS Ray, I recall you asking at some point not long ago about copying
one disk to a disk of a different size.  In Rute User's Guide he writes
"If they (disks) are not the same size, you will have to use tar or
mirrordir to replicate the file system exactly."  I guess you got that
task accomplished, but did you do it using this means, or by some other?
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: writing disk image to file

[Ray Olszewski]

At 10:53 PM 6/8/2003 -0500, James Miller wrote:
>I've got a small disk whose contents I'd like to save as an image file, in
>case I ever want to write it back to the disk it came from.  This is a 4MB
>flash disk.  From what I gather in my Linux reading, I should do something
>like the following: hook this tiny IDE disk up as, say /dev/hdb and the do
>something like "dd if=/dev/hdb of=dev-hdb-image.img" ? ? ? (the question
>marks outside the quotes indicates things I'm not sure about).  This will
>write an image file to, I suppose the current dir - correct?

Yes. As always, there are unstated assumptions ... for example, the user 
executing the dd command needs read access to the hdb device (not its 
filesystem, the block device itself) and write access to the pwd (= 
"present working directory", Linux/Unix jargon for what you call "the 
current dir"). But these unstated assumptions are nothing special ... 
though in practice they usually mean you need to run dd as root.

>  Is the
>output file's name somewhat a matter of indifference, as it seems to me?
>E.g., wouldn't necessarily require any extension (might be called simply
>"dev-hdb-image"), though an extension might help me keep track of just
>what sort of file it is.

Unlike Windows, in Linux (and Unix) extensions have no special meanings to 
the system; they are just part of the filename, there for the convenience 
of the user. There are conventions -- .c means a C source program, .pl a 
Perl program, .cpp a C++ program, .sh a shell script ... but these are 
conventions only. Though some applications (like the gcc compiler) enforce 
some extension rules about the files they work with, there is nothing here 
that corresponds to Windows using the extension to identify the application 
that is used to open a file.

Pick whatever you want to ... though .img is an extension commonly used for 
this sort of image file (typically in the context of floppy images).

>And, finally, the question marks: I see
>sometimes "bs=512" or some such following the proposed file name which, as
>I get it, refers to block size.  I'm not really clear on block sizes and
>what they do, but I have run accross information indicating that, in the
>case of an image file, it's not too important and can be left out.  Can
>anyone illumine my benightedness on this matter?

Picking a block size that matches the size of the physical device in the 
transfer (in this example, that would be hdb) can speed up the transfer 
process. But I haven't seen it as having any other practical significance, 
and even that effect is small with today's fast systems.

>Thanks, James
>
>PS Ray, I recall you asking at some point not long ago about copying
>one disk to a disk of a different size.  In Rute User's Guide he writes
>"If they (disks) are not the same size, you will have to use tar or
>mirrordir to replicate the file system exactly."  I guess you got that
>task accomplished, but did you do it using this means, or by some other?

At least for the system I was cloning, doing a dd worked fine, with the 
destination drive being larger than the source drive. I was later able to 
use fdisk to create additional partitions in the unused area of the 
destination drive.



-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: writing disk image to file

[Stephen Samuel]

James Miller wrote:
> I've got a small disk whose contents I'd like to save as an image file, in
> case I ever want to write it back to the disk it came from.  This is a 4MB
> flash disk.  From what I gather in my Linux reading, I should do something
> like the following: hook this tiny IDE disk up as, say /dev/hdb and the do
> something like "dd if=/dev/hdb of=dev-hdb-image.img" ? ? ? (the question
Yep. That would work fine as is.

.....
> what sort of file it is.  And, finally, the question marks: I see
> sometimes "bs=512" or some such following the proposed file name which, as
> I get it, refers to block size.  I'm not really clear on block sizes and
> what they do, but I have run accross information indicating that, in the
> case of an image file, it's not too important and can be left out.  Can
> anyone illumine my benightedness on this matter?

Block sizing indicates how big of a chunk DD will read and write the data
in (dd can also split them up into ibs=___ and obs=___ -- in which case it
will create separate input and output buffers of different sizes).

The main purpose of the BS parameters was to take care of I/O devices for
which the write size made a difference. tape drives with variable block
sizes or streaming come to mind. Depending on the tape drive, you could
sometimes end up with very different performance and even effective tape
capacities depending on how big your write block sizes were.

In the current day, the biggest value to bs= is taking advantage of the
HD's caching capabilities. I.E. sometimes reading in chunks as big as (or
much bigger than) the track/cylinder size can result in a notable performance
increase.  It also cuts down on system overhead.

My quick test (below) showed a barely noticable performance increase, but a
more noticable cut (percentage wise) in terms of CPU usage.  In this case,
I'm comparing a 512byte buffer with a 5meg buffer:

Note that I'm testing read performance only, not write perfromance (no spare
disk partitions to mess with)
[root etc]# time /usr/bin/time  dd if=/dev/hda5 of=/dev/null bs=512
2056256+0 records in
2056256+0 records out

real    1m12.906s
user    0m2.430s
sys     0m11.050s

[root etc]# time /usr/bin/time  dd if=/dev/hda5 of=/dev/null bs=5120k
200+1 records in
200+1 records out

real    1m12.585s
user    0m0.010s
sys     0m10.020s

In this case, the 10000 times factor in writes resulted in the user CPU time
dropping from 2.4 seconds to 10miliseconds while the system CPU consumption
only dropped by 10%

> 
> Thanks, James
> 
> PS Ray, I recall you asking at some point not long ago about copying
> one disk to a disk of a different size.  In Rute User's Guide he writes
> "If they (disks) are not the same size, you will have to use tar or
> mirrordir to replicate the file system exactly."  I guess you got that
> task accomplished, but did you do it using this means, or by some other?

What will happen if you copy (say) a 1GB partition to a 4GB partition
is that it will more or less work, but you still have a filesystem that
has data indicating that there's only 1GB of disk space in the partition --
in other words, it will ignore the extra 3GB of disk space. The main
solutions to that problem are either:

1) create (mkfs) a new (empty) partitions and then bit copy the FILES (at the
    user level) onto the new partition (e.g. with tar), or
2) bit-copy the partition over, and then manipulate the File System data directly
    to be consistent with the extra available disk space. parted is the tool that
    I usually use for things like that.

-- 
Stephen Samuel +1(604)876-0426                samuel@bcgreen.com
		   http://www.bcgreen.com/~samuel/
Powerful committed communication, reaching through fear, uncertainty and
doubt to touch the jewel within each person and bring it to life.

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: writing disk image to file

[Mike Dresser]

On Tue, 10 Jun 2003, Stephen Samuel wrote:

> The main purpose of the BS parameters was to take care of I/O devices for
> which the write size made a difference. tape drives with variable block
> sizes or streaming come to mind. Depending on the tape drive, you could
> sometimes end up with very different performance and even effective tape
> capacities depending on how big your write block sizes were.

Think this would explain why I only get about 16.5-17 gig out of a DDS-4
tape drive using the default block options (in other words, i didn't
specify) on tar?

I know some of it is marketting, that they don't _ever_ get 20 gig out of
them for various reasons.

Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs