"resume" cp/mv

"resume" cp/mv

[urgrue]

Anyone know of any trick to make mv/cp support some form of crash-recovery?

The problem is, if I'm moving a huge directory/file, and the transfer 
fails, there is absolutely no way to continue where I left off. This is 
awfully primitive.

The only way I can think of is to zssh into my own localhost and use 
rz/sz, or the same via ftp, but this seems like a pathetic kludge, and 
even those dont work very well with large directories full of files.

Re: "resume" cp/mv

[Thornton Prime]

On 2/6/06, urgrue <urgrue@bulbous.org> wrote:
> Anyone know of any trick to make mv/cp support some form of crash-recovery?

Use rsync.

thornton

Re: "resume" cp/mv

[Tim Walberg]

Sounds like a good match for rsync...


On 02/06/2006 17:28 +0200, urgrue wrote:
>>	Anyone know of any trick to make mv/cp support some form of crash-recovery?
>>	
>>	The problem is, if I'm moving a huge directory/file, and the transfer 
>>	fails, there is absolutely no way to continue where I left off. This is 
>>	awfully primitive.
>>	
>>	The only way I can think of is to zssh into my own localhost and use 
>>	rz/sz, or the same via ftp, but this seems like a pathetic kludge, and 
>>	even those dont work very well with large directories full of files.
>>	-
>>	To unsubscribe from this list: send the line "unsubscribe linux-admin" in
>>	the body of a message to majordomo@vger.kernel.org
>>	More majordomo info at  http://vger.kernel.org/majordomo-info.html
End of included message



-- 
+--------------------------+------------------------------+
| Tim Walberg              | twalberg@mindspring.com      |
| 830 Carriage Dr.         | www.mindspring.com/~twalberg |
| Algonquin, IL 60102      |                              |
+--------------------------+------------------------------+

Re: "resume" cp/mv

[Scott Taylor]

On Mon, February 6, 2006 07:28, urgrue wrote:
> Anyone know of any trick to make mv/cp support some form of
> crash-recovery?
>
> The problem is, if I'm moving a huge directory/file, and the transfer
> fails, there is absolutely no way to continue where I left off. This is
> awfully primitive.
>
> The only way I can think of is to zssh into my own localhost and use
> rz/sz, or the same via ftp, but this seems like a pathetic kludge, and
> even those dont work very well with large directories full of files.

Maybe you want to try cpio?  Less likely to fail then mv or cp on large
trees.  As with mv or cp you skip files that already exist, no need to
start over.

--
Scott

Re: "resume" cp/mv

[hansel]

I've used 'cp -u' for a similar problem (where I really am updating
parallel --backup-- directories of huge graphics files).

Mark Hansel

On Mon, 6 Feb 2006, urgrue wrote:

> Anyone know of any trick to make mv/cp support some form of crash-recovery?
> 
> The problem is, if I'm moving a huge directory/file, and the transfer 
> fails, there is absolutely no way to continue where I left off. This is 
> awfully primitive.
> 
> The only way I can think of is to zssh into my own localhost and use 
> rz/sz, or the same via ftp, but this seems like a pathetic kludge, and 
> even those dont work very well with large directories full of files.
> -

Re: "resume" cp/mv

[Prosenjit Kundu]

I have also used 'cp -u'.......but rsync is better match for it.

PK

On 2/6/06, hansel@hansel.mnstate.edu <hansel@hansel.mnstate.edu> wrote:
> I've used 'cp -u' for a similar problem (where I really am updating
> parallel --backup-- directories of huge graphics files).
>
> Mark Hansel
>
> On Mon, 6 Feb 2006, urgrue wrote:
>
> > Anyone know of any trick to make mv/cp support some form of crash-recovery?
> >
> > The problem is, if I'm moving a huge directory/file, and the transfer
> > fails, there is absolutely no way to continue where I left off. This is
> > awfully primitive.
> >
> > The only way I can think of is to zssh into my own localhost and use
> > rz/sz, or the same via ftp, but this seems like a pathetic kludge, and
> > even those dont work very well with large directories full of files.
> > -
> -
> To unsubscribe from this list: send the line "unsubscribe linux-admin" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


--
Regards,
Prosenjit Kundu

Re: "resume" cp/mv

[urgrue]

rsync certainly helps with large directories, so thats at least a 
partial solution, but it doesnt resume an interrupted transfer of an 
individual file, does it?

Prosenjit Kundu wrote:
> I have also used 'cp -u'.......but rsync is better match for it.
> 
> PK
> 
> On 2/6/06, hansel@hansel.mnstate.edu <hansel@hansel.mnstate.edu> wrote:
>> I've used 'cp -u' for a similar problem (where I really am updating
>> parallel --backup-- directories of huge graphics files).
>>
>> Mark Hansel
>>
>> On Mon, 6 Feb 2006, urgrue wrote:
>>
>>> Anyone know of any trick to make mv/cp support some form of crash-recovery?
>>>
>>> The problem is, if I'm moving a huge directory/file, and the transfer
>>> fails, there is absolutely no way to continue where I left off. This is
>>> awfully primitive.
>>>
>>> The only way I can think of is to zssh into my own localhost and use
>>> rz/sz, or the same via ftp, but this seems like a pathetic kludge, and
>>> even those dont work very well with large directories full of files.
>>> -
>> -
>> To unsubscribe from this list: send the line "unsubscribe linux-admin" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> 
> --
> Regards,
> Prosenjit Kundu
> -
> To unsubscribe from this list: send the line "unsubscribe linux-admin" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: "resume" cp/mv

[Jeff Woods]

At 17:50 +0200 2/7/2006, urgrue wrote:
>rsync certainly helps with large directories, so thats at least a 
>partial solution, but it doesnt resume an interrupted transfer of an 
>individual file, does it?

rsync compares file size and timestamp(s?) and if they are the same 
it presumes the contents are the same. If they differ it then reads 
the file and generates checksums for each chunk of the file, compares 
the checksums (much faster network load than sending the data) and 
only updates the chunks that don't have a matching checksum. This 
implies that it does not resend (or rewrite) the bulk of the file 
that's consistent, but that it has to read all of both the source and 
destination file. If you're rsync'ing over a network the performance 
is reasonably close to resuming where it left off but it doesn't 
depend on maintaining the state of an aborted copy.

Note: Options exist to override or tune several of the details above, 
but I believe what I described is essentially the normal rsync behavior.

--
Jeff Woods <kazrak+kernel@cesmail.net>